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I. INTRODUCTION 

Over the last few years, federal, state, and local Interest has focused 
on improving classroom and school activitiek through the application of 

process-product research on classroom and school ef fi^ctiveness. The 

f' 

organizational structures and delivery systems used in these endeavors are 
sometimes influenced by the research on planned change for school improvement. 

In Maryland, application of such research is apparent in the School 
Improvement Through Instructional Process (SITIP) program, initiated in 1980. 
All 24 local educational agencies (LEAs) voluntarily -iraplemented one or more 
of four research-based instructional improvement models: Active Teaching, 
Mastery Learning, Student Team Learning, and Teaching Variables. They were 
supported in thei^ -f forts by the Maryland State Department of Educatioii 
(MSDE), and both local and state staff were encouraged to design and carry out 
their tasks in ways suggested by the research on planned change. 

In order to facilitate data-based decision-making with respect to SITIP, 
evaluation was conducted In such a way as to provide relevant information in a 
timely fashion. Three major reports were developed: the first focused on 
implementation, the second on impact, and the present report focuses on 
institutionalization.* 

This report begins with an overview of SITIP, and discusses: evaluation, 
state Initiatives, and local implementation. The final chapter presents a 
summary and conclusions. 



* These publications and others describing SITIP activities are listed in the 
bibliography of this report. 



II. OVERVIEW OF MARYLAND'S SCHOOL IMPROVEMENT PROGRAM 

V; 

This chapter presents an overview of Maryland's current school improve- 
ment program, outlines the four instruct l^on^l models used in the program, and 
summarizes the results reported for the period December 1980 to June 1983. 

The SITIP Design 

Maryland's School Implrovement Through Inetruct'ional Process (SITIP) . 
program involves local education agencies (LEAs) in voluntarily implementing 
instructional processes proven to be effective in increasing student achieve- 
ment. The instructional models used in SITIP are: Active Teaching, Mastery ^ 
Learning, Student Team Learning, and Teaching Variables, All four models are 
research-based and were selected by the Maryland State Department of Education 
(MSDE) as potentially useful to all schools for improving Instruction in all 
structured academic curricula. During the 1981-82 school year, nearly 700 
teachers in grades K-12 used one or more of the models in their instruction of 
mathematics, reading/language arts, science, social studies, or other academic 
areas. During the 1982-83 school year* over 980 tfiachers in 139 schools were 
aimilarly involved. 

SITXj? was designed by MSDE as a multi-year program consisting of inter- 
active activities which are outlined below and presented in Figure I. 

1. Preparation (open systems planning): Identify needs and potential 
solutions. Draft the SITIP design. 

2. Initial Commitment ; Review plan with LEA superintendents. Obtain 
approval for local t^ams to attend av^areness conferences. 

3. Awareness Conferences : Have each of the instructional improvement 
models presented by its developer at awareness conferences attended 
by LEA teams, MSDE staff, and interested others. Describe the 
design and -nature of (voluntary team) Involvement. 



^. Local PropOTals/Plans : Help cross-hierarchical local teams draft 
proposals to implement one or more of the models. 
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NSDC Preparation 
(opan-ayatana planning) 



Initial Comnicmenc 
(of local teams) 



Awareness Conferences 
(content and deslf;n 
information) 



\ 



Ttchnlcal Assistance 
(coaching) 



Local Proposals and Plana 
(with allowance for semi- 
annual or annual up-nlatea) 



FolVyw-up Training 



lAplenencacion 



Assessment of Progress 
and Impact 




Diaaeminacion 
(expanaion) 



laat itMt iMaIiB»i^on 
Taniaatldii 



Figure 1. The SITIP Design: An InteraTltive Model for Program Improvement 
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5* Implementation : Help LEAs implement selected models using their ovm 
strategies but involving representatives of all role groups. 

6. Dissemination : Encourage use of the models In many schools, and 
share information about successes between LEAs. 

7. Technical Assistance : Assign MSDE staff (across divisions) to 
assist LEAs in planning^ implementation » and dissemination; to 
conduct follow-ups; and to facilitate networking. Build capacity; 
do not create dependency. 

®* Training : Conduct an intensive three-day training session on each 
model for prospective implementers (teachers, school administrators, 
central office staff). Conduct annual or semi-annual follow-up 
training sessions (using participatory planning) to maintain quality 
implementation. Assist local staff in planning/conducting turnkey 
training. 

9. Assessment of Progress and Impact (cyclical): Have^a "third party 
evaluator" collect and analyze data systematically and use (feed- 
back) information to make improvements and publicize successes. 

10. Inst itutionalizat ion/Termination : Integrate SITIP activities into 
local programs (independent of state support) or terminate, using 
locally relevant data to make decisions.* 

The activities outlined above began in 1980. All 2A LEAs In the state 
were represented at awareness conferences. Nineteen LEAs submitted proposals 
for implementation through June 1983. For the 4982-8A school years, five 
"new" (second wave) LEAs decided to participate and for the 1983-8A year one 
"veteran" LEA did not request state funds. State department support (funding, 
provision of training and technical assistance to LEAs) continued through the 
1983-84 school year. At that time it was hoped that LEAs would terminate or 
institutionalize their model programs, with each district taking responsibi- 
lity for local needs, decisions, and actions. "Matching" grants were 
available after July 198A only for second wave LEAs or for program expansion. 
A chronology of key activities is presented in Table 1. The cycle of 
planning, training, implementation, and evaluation Is a consequence of the 
SITIP design. 



* This element of the design was added in August 1983 as LEAs began to provide 
matching funds to (reduced) state grants. 
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Table I 

Chronology of Key SITI; \ctlvitles 





1980 

1 me Sept Dec 


1981 

March June Sept Dec 


1982 

March June Sept Dec 


1983 

March June Set>t Dec 


1984 

March June Sept Dec 


1. MSDE preparation begun 

2. Commltmer: given by LEA super- 
intendents & developers re. 
awareness conferences 

3. Awareness conferences held 


X 

X 

I 


-I 








4. Local proposals submitted to 
MSDE 

5. MSDE/LEA/devcloper planning of 
suauaer Institutes 

6. MSDE technical assl|§ance 
provided 

V. Four suoner Institutes held 
8. MSDE/LEA p^lannlng modified 




I— I 
t 1 








X 

X 








9. Implementation (by 19 LEAs) 

10. Follow-up training conducted 

11. Instructional Leadership 
conferences conducted 

12. Evaluation report on Imple- 
mentat on distributed 




X 


XKXXX 

X 

X 


X 


X 


13. Plannf.ng and orientation for 

new LCAs 

14. Implementation (by 5 "new" 
LEAs) 

15. Follow-up training conducted 

16. Combined summer Institute held 

17. Local plans finalised 






t T 

XXX 


X X 

X 

X 




18. [valuation report on Impact 
distributed 

19. Follow-up training conducted 

20. MSDE/LEA planning reviewed 

21. Local "ownership" for institu- 
tionalization or termination 

22. Evaluation report on institu- 
tionalization distributed 






X 

XK 


XX 

I— I 

X 
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The Instructional Models 



Four research-based instructional models (innovations) were selected by MSDE 
as potentially useful to all schools for improving instruction In all 
structured academic curricula. They were: Active Teaching, Mastery Learning, 
Student Team Learning, and Teaching Variables. 

• Active Teaching (AT) is a system of direct instruction developed by 
Thomas Good and Douglas GrouWL at the University o^ Missouri. 
Originally designed for the teaching of mathematics, AT consists of 
the following components: 

1. Pre-lesson development — concepts and skills from the previous 
night's homework are reviewed, homework is checked and collected, 
and students engage in mental exercises. 

2. Lesson development ~ prerequisite skills and concepts are briefly 
reviewed, new concepts are introduced via teacher explanation and 
demonstration. 

3. Controlled practice. 

A. Independent, uninterrupted, individual, successful practice is 
provided in order to increase proficiency in the skills and 
concepts taught. 

5. Homework — the homework that is assigned is related to the 
concepts developed that day. 

6. Review/maintenance — weekly and end-of-unit reviews help to 
maintain skills and concepts taught. 

• Mastery Learning (ML), developed by Benjamin Bloom (University of 
Chicago) and James Block (University of California), combines 
curriculum alignment and diagnostic/prescriptive instruction with a 
philosophy that all students can succeed. Essential components are: 

1. Developing a scope and sequence of objectives, brokei) down into 
prerequisite and component skills. 

2. ^Providing appropriate instruction aligned with the objectives to 

be mastered. 

3. Testing the student's progress in mastering the objectives through 
the use of a formative evaluation measure ("no fault" test). 

A. Providing students who have not achieved mastery with .iddltlonal 
corrective work in the de icient ar^as specified by the formative 
tests, and providing students who have achieved mastery with 
enrichment activities to reinforce and supplement learning. 
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5. Testing final mastery of the objectives with a suraraative evalua- 
tion measure. 

6, Recording student progress in terms of individual mastery of 
specific objectives. "Mastery" is usually defined as 80% of the 
students demonstrating success on at least 80% of the objectives 
in a given unit of instruction. 

• Student Team Learning (STL) techniques use peer tutoring and team 
competition to facilitate student learning. Student Team-Achie^vement 
Divisions (STAD) and Teams-Games-Tournaments (TGT) were developed by 
Robert Slavln and staff at the Johns Hopkins University. Jigsaw was 
started at the University of California at Santa Cruz. The key 
factors of STL are peer interaction, cooperation, and competition. 
STAD is basically team learning; TGT is team learning plus competi- 
tion by ability level; Jigsaw Is team learning of specific elements 
of a program, with regrouping for peer teaching across elements. 

• Teaching Variables (TV) was developed by David Helms and staff at 
Research for Better Schools (RBS) . Two variables found to be strongly 
related to effectiveness of instruction and student achievement were 
identified: "content" and "time." The "content" variable encompasses 
two factors: 

1. Assessment of prior learning. 

2. Alignment of curriculum objectives and classroom instrction to 
the testing instrument. 

The "time" variable improvement cycle involves; 

1. Measuring student engaged time (SET) via classroom observation. 

2. Comparing SET and opportunity for improvement. 

3. Reviewing and selecting research-based improvement strategies. 

4. Implementing strategies. 

5. Using additional classroom observations to evaluate the 
effectiveness of the strategies in improving SET. 

The innovations vary in complexity. Complexity was detenpined on four 
riteria: 

• knowledge — how much must be learned that is new? 

• materials — how much do classroom materials need to be redesigned 
or developed? 

• methods — how much change is required in the way things are done in 
the classroom? in the school? 

• organization how much role change and administrative action are 
required? 
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Each innovation was assigned a rating on a scale from 1 to 5 (with 5 
indicating high complexity) on each criterion, and a mean rating was derived. 
As designed, the innovations were rated in order of complexity as: Active 
Teaching (1.62), Student Team Learning (2.37), Mastery Learning (3.12), and 
Teaching Variables (3.75). (See Table 2.) 

As implemented by some. Teaching Variables proved to be less complex than 
Mastery Learning since 60% of TV implementers used only the "time" variable. 
AT and STL, as implemented, were simple and classroom-based, requiring less 
support from school administrators and central office staff than ML or TV. ML 
and TV were both complex and required cross-hierarchical coordination for 
successful implementation. 



Table 2 

Complexity of the SITIP Innovations 





Models 


Dimension 


AT 


ML 


STL 


TV 


Knowledge 


2 


3 


3 


5 


Materials 


2 


4 




3 


Methods - in class 
- in school 




4-\3.5 
3/ 




ir 


Organization 


1 


2 


1 


b» — 












Total 


6.5 


12.5 


9.5 


15 


Mean 


1.62 


3.12 


2.37 


3.75 



Assigned ratings vary from a high of 5.00 to a low of 1.00. 
AT - Active Teaching, ML - Mastery Learning, 
STL - Student Team Learning, TV - Teaching Variables 
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Summary of Findings; December 1980 to vJune 19 83 
The following discussion is summarized from earlier SITIP reports that 
focused primarily on implementation and impact. 

The nature of the innovation is one of many factors influencing imple- 
mentation. Other influential factors include the choice of strategy,* the 
nature and extent of training and assistance, extent of local commitment, and 
the nature of involvement. In comparing in^vations, the following should be 
kept in mind: 

• Active Teaching * The strategies chosen required active involvement 
from all role groups. The innovation as implemented was simple and 
classroom-based. The scope of implementation was larger than for any 
other model (33 schools, 472 teachers in June 1982; 73 schools, 581 
teachers by June 1983). 

• Mastery Learning . The strategies chosen were school-based. The 
innovation as implemented was complex and suggested a need for 
cross-hierarchical coordination. The scope of implementation was 
moderate (81 teachers in six schools in June 1982; 202 teachers in 13 
schools by June 1983). 

• Student Team Learning . The strategies chosen were primarily 
teacher-oriented or classroom-basec* with initial involvement or light 
monitoring by school administrators and central office staff. The 
innovation as implemented was fairly simple and classroom-based. The 
scope of implenentation was moderate (100+ teachers in 20+ schools in 
June 1982; 113 teachers in 42 schools by June 1983). 

• Teaching Variables . The strategies chosen were primarily school-based 
with active involvement by central office staff in three of the five 
LEAs. The innovation as Implemented was moderately complete suggesting 
a need for interaction between observers and teachers obseryed. The 
scope of implementation was low (50+ teachers In six schools in June 
1982; 98 teachers in 13 schools by June 1983). 

Regardless of the model adopted, it was found that certain roles and 

responsibilities were effective in facilitating instructional improvement. 



* Strategies of implementation were designed by LRAs and included: ^ light- 
house school approach, capacity building through staff development:, a pilot 
school to district design, and district-wide implementation. 
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• SEA staff initiate, encourage voluntary participation, build and 
maintain comraltment , and provide (research-based) assistance as 
resource coordinators. 

• Central office staff engage in cross-hierarchical conununication, ^ 
linking schools and LEA to tne SEA, and act as resource coordinators 
by providing various support services. If implementation is in more 
than one school, central office staff function as "project directors." 

• School-based administrators ensure that teachers' concerns are 
addressed (logistical and affective), and function as supportive 
facilitators or managers, sometimes with "project director" status 
if a "lighthouse school" strategy is used. 

• Teachers carry out classroom implementation tasks. Also, teacher 
representatives support others by "turnkey training," especially for 
capacity building sites, and, when implementation Is single-school 
focused, teachers can function as "project directors" if administra- 
tors (school or central office) do not take on that responsibility. 

Other findings of this study, relevant to role group responsibilities in 

implementation, suggest the following conclusions. 

• Initial staff interest or commitment to implement a new program or 
practice can be built if: (1) the superintendent permits staff to 
look at new ideas with the intent to implement if appropriate, (2) 
the innovation and its presenter/developer have validity and credi- 
bility, and (3) staff believe that they do have choices and can 
influence decisions. 

• Staff interest is the most important factor in selection of the 
innovation and in determination of elements of the implementation 
plan. 

• Cross-hierarchical planning facilitates mutual understanding which 
helps to prevent problems during implementation (such as communica- 
tion breakdowns, resentment, feelings of isolation). 

• Representation of the various role groups in planning and subsequent 
decision-making builds understanding and commitment, ensures inclusion 
of role group perspectives, and strengthens organizational knowledge 
so that knowledge is not lost if reassignments are made, and new staff 
are not given one-sided briefings. 

• The complexity of the innovation determines the amount of work to be 
done for a given school site. 

• The implementation strategy determines how the work is shared among 
role groups and how the burdens shift among role groups over time. 
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The strategy plus the scope of implementation (number of schools, 
teachers, curricular subjects, and grade levels; amount of time 
devoted to the innovation for each class or subject) determine how 
much work is to be done within a given LEA. 

The nature and extent of communication and decision-making determine 
productivity and effect* 

The organizational norms of the LEA determine communication and 
decision-making procedures. 

Changes in organizational norms are Influenced by internal and 
external pressures which act virtually simultaneously, but not 
necessarily collaboratively. 

Regardless of the nature of the innovation, all role groups must 
carry out the following tasks, in order of investment: (1) inter- 
active support (acknowledgement, shared knowledge, problem-solving, 
resource allocation); (2) learning/training (before and during 
implementation); (3) record-keeping; (4) materials identification or 
development; (5) evaluation; and (6) administration* 

Perceptions of interactive support reflect participants' assessment 
of each others' commitment. Judgments are based both on how much 
useful help was provided and on the visibility of the support (with 
lower ratings for low visibility). 

It is preferable for each role group to perceive high support from 
close role groups rather than distant ones. Therefore, visibility 
should be reduced with distance. [For instance, teachers should 
perceive principals as supportive. If there is a problem, a state 
technical assistant may help central office staff (who turnkey ideas 
to the principal) or the state technical assistant (TA) (with central 
office permission) may help the principal. However, the state TA does 
not provide support to the teachers when it should more appropriately 
come from the principal.] 

RepresentativfiS of all role groups need a thorough understanding of 
innovations to be adopted so. that: (1) plans are realistic, (2) 
reassignments do not result in the organization's loss of knowledge, 
(3) interactive support can occur, (4) no one group is overburdened, 
and (5) there is a reasonable chance for institutionalization and 
dissemination beyond initial pilot sites. 

Impact in terms of student achievement was evident to some extent, 
although not formally expected for the first year of implementation. 
Results suggest (tentatively) that greatest impact was made by 
Mastery Learning, followed by Active Teaching. Student Team Learning 
appeared to influence student affect more than achievement. 
Teaching Variables data are inconclusive. 

People providing technical assistance (TA) are most effective when 
they are: (1) responsive to the needs of the implementers, (2) task 
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oriented and knowledgeable about local norms, the innovation, and 
processes of planned change, and (3) skillful in facilitating shared 
decision-making and in coordinating communication. AccompilFhments of 
the TA team included: providing leadership for a statewide schiol 
improvement program while concurrently encouraging local ownership; 
maintaining communication within MSDE and among LEAs; developing 
networks and teaching/learning opportunities for local teams to share 
successes and build expertise; developing erxpertise among themselves 
and applying it in SITIP and in other areas; and increasing awareness 
of effective SITIP practices among researchers and educators outside 
Maryland. Impact relating specifically to assistance included': 
increased trust and openness in communication between LEAs and MSDE; 
increased effort by some LEAs to carry out their plans; better linkage 
or a clear common knowledge among hierarchical levels within LEAs; 
increased involvement by central office staff in some LEAs; changes in 
planning, decision-making, and/or communication (e.g., more 
involvement of teachers) in some LEAs. 

Major outcomes of planning activities within MSDE (other than- the 
implementation of the plans) included: (1) a general knowledge of 
SITIP by most MSDE staff; (2) sufficient commitment or interest by 
senior and middle management to be willing to explore elements or 
knowledge bases of SITIP, and to continue (and expand) cooperative 
support for technical assistance; (3) application of SITIP-related 
information, strategies, or processes in various existing programs; 
and (4) increased knowledge and skills in instructional improvement 
and planned change by members of the TA team (which Informally 
filtered back into other program areas) . 

The various kinds of training events reinforced each other, and helped 
to establish a common knowledge base for all hierarchical levels. 
Participant evaluation of events, the subsequent local requests for 
on-site presentations and assistance, and the scope and fidelity of 
implementation, provided strong evidence of the value to . -rticipants 
of the SITIP-related training provided by MSDE. 

While processes of implementation based on the research on planned 
change were recommended for all models in all LEAs, and TAs encouraged 
local educators to attend to such principles as participatory ' < 
decision-making, two-way communication, training and support, and 
appropriate investment of time and energy, those processes of imple- 
mentation and principles were not always applied. When they were 
applied, implementation went sufficiently smoothly for energy to move 
gradually from establishing structures, relationships, and 
expectations toward actual classroom use. When there were arbitrary 
administrative decisions, top-down or incomplete communication, low 
support by central office staff, and insufficient time allocated for 
materials development or group planning by teachers, implementation 
problems occurred. 

Impact was made on student achievement by three models (AT, Ml,, STL), 
with the strongest evidence of success In mathematics and reading/ 
language arts for AT and ML. (Standardized test results were reported 
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by four projects, and results of teacher-made tests were reported by 
eight projects,) Positive results were most apparent when AT or ML 
was used consistently over a period of time for a given subject and 
grade. 

• Impact was made on student attitudes to some extent by all models. 
Data summarized for 12 projects (AT, ML, STL) indicated that SITIP 
students enjoyed the lessons, did not find them difficult, and wanted 
to succeed. Friction among students and their perceptions of 
favoritism and disorganization needed to be addressed at some sites. 
While teachers believed that STL students' self-esteem and willingness 
to work with others increased, student data for STL indicated no 
differences for that model in comparison to AT and ML. 

• Impact was made on teachers' knowledge by the training provided in 

• support of all models. Skills in new teaching/observation techniques 
increased through classroom practice and coaching. Positive attitudes 
about teaching were strengthened as teachers experienced success. 

• Impact was made op a school (the faculty and how instructional matters 
were dealt with' .hrough commitment and sharing among teachers (ML, 
STL, TV), and provision of support (ML) and recognition of success 
(STL) by school administrators (usually the principal). Staff 
interest in teaching/learning increased (AT, STL); there was more 
continuity across classes (AT); better management of instruction (TV); 
and closer monitoring of teaching (4Jy~ 

• Impact was made on school administrators' knowledge of all models 
through training, and they itnproved in instructional management (AT), 
strengthened their belief in traditional teaching (AT), and were more 
appreciative of teachers' capabilities (AT, ML) as Implementation 
occurred in their schools. 

• Impact was made on central office staffs' knowledge of all models 
through training, and, they improved in instructional management as 
they became involved in implementation (AT). 

• At the system level, there was knowledge gain (STL), cross-hier- 
archical sharing and commitment (AT, ML), and policies enacted to 
release teachers to train others or coordinate activities XML), and to 
implement the model district-wide for a given subject or grade level 
(AT) . 

• Concerns were stated and recoimnendations made by local SITIP 
participants at the end of the 1982-83 school year. For AT, the most 
critical issue vas local perceptions of the fit of the model to 
specific grades, subjects, or students (as grouped). For ML, the most 
critical issue was cost-effectiveness in terms of time allocated for 
unit and test development, and the subsequent record-keeping, in 
relation to the perceived value of the model. For STL, the most 
critical issue was cost-effectiveness In terms of teachers' investment 
in relation to impact (including discipline) on various kinds of 
students. For TV, the most critical issue was the perceptions — 
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fear, apathy, resentment (primarily of teachers) — about local 
implementation decisions and about the model design. While those 
issues suggest negative impact in some sites, it should be noted that 
they were not pervasive and did not out-weigh the positive impacts 
reported earlier. 

Designs or plans for instructional improvement are most likely to b« 
successful if: (1) participation (of organizations) is voluntary, (2) 
comrnunica^tion is multi-dimensional, (3) planning is interactive with 
training, (4) training and technical ^assistance are provided during 
implementation, (5) ''lip service compliance" Is not accepted as imple- 
mentation, (6) adjustments of scope are considered legitimate if they 
relate to resources available, and (7) each participant has some 
degree of choice about his or her involvement (nature or extent) in 
the effort. These elements weie present in SITIP. 
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'III. EVALUATION OVERVIEW 

The evaluation was designed to address two "levels:" (1) specific events 
or stages of activity, and (2) the overall SITIP program as a viable strategy 
for statewide school improvement. Also, MSDE required the findings to be 
reported on an on-going basis so that data-based decisionr could b^p made to 
bring about program improvements. During the 1980-1981 and 1981-1982 school 
years, an evaluation of the overall implementation of SITIP was conducted for 
MSDE by Research for Better Schools (RBS). During the 1982-1983 school year, 
RES continued to take primary responsibility for evaluation, but LEAs assumed 
additional responsibilities, and "impact" data received greater attention. 
During the 1983-84 school year implementation and impact continued to be 
examined, but the primary focus shifted to institutionalization. 

This chapter summarizes the questions addressed by the evaluation during 
the 1983-84 school year, and the responsibilities and data sources, measures 
and methods, and data analysis and reporting procedures employed in the study. 

Questions Addressed 

The study addressed five areas: institutionalization, impact, implemen- 
tation, dissemination, and technical assistance. 

1. What is the nature and extent of institutionalization: 

1.1 At the state level? 

1.2 Across LEAs for a given model? 

1.3 Within a local system? 

1.4 Within a school? 

2. What is the nature and extent of impact: 

2.1 On educators, schools, and LEAs? 

2.2 On student achievement, behavior, and attitudes? 
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3, Wnnt is the nature and extent of implementation: 

3.1 Within a local system? 

3/2 Across LEAs implementing a given model? 

A. What is the natute and extent X>i dissemination: 

4.1 Within a local system? 

A. 2 Between LEAs? 

A. 3 Outside of Maryland? 

5. What is the nature and extent of technical assistance prpvided by MSPE 
in terms of : ^ \ ' 

^■ 

5.1 Implementation? « 

5.2 Dissemination? 

5.3 Institutionalization? 

5. A Evaluation ? . ' 



Responsibilities and Data Sources 
RBS worked with MSDE TAs to develop an overall design and written 
guidelines for evaluation* The guidelines summarized the design, listed role 
group and individual responsibilities, included a checklist planning sheet 
which indicated mandates (e.g., completlpn by local implementers df specific 
questionnaires), and described measures and methods. RBS and MSDE staff 
reviewed the guidelines with project teams and each t^am completed a planning v 
sheet agreeing to a coordinated evaluation effort.* 

In general, RBS was responsible for design, development, analysj^, and 
reporting. MSDE TAs were responsible for coordination, distribution and '\ 
collection of materials (e.g., questionnaires), and assistance to LEAs in 
following the guidelines (e.g., selection and use of student assessment 
measures). LEA coordinators carried out tasks similar to those of TAs,' but 
each In his/her own district. 



* There are 2A LEAs In Maryland, all of which participated In SITIP. Since 
some LEAs adopted more than one Instructional model, there were 29 projecCls. 
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Information — materials, interviews, survey responses was provided 
by: (1) the seven MSDE TAs and the SEA assistant deput/ superintendent; (2) 
LEA central office staff directly involved in SITIP (usually one to three 
people for each of 24 LEAs); (3) school-based administrators (up to 3 per 
project); and (4) teachers (up to 15 per project). On-site visit information 
wars provided by additional educators. Also, large numbers of participants of 
state-sponsored training events provided information either directly 
(responding to questionnaires) or indirectly (observed by RBS). In some cases 
student data were summarized by local educators and submitted to RBS. 

Measures and Methods of Data Collection 
Five general methods of data collection were used: observations, inter- 
views, questionnaires, document analyses, and measures of student attitudes 
and achievement. 
Observations 

The evaluation design included observations by RBS staff of state- 
sponsored training events, MSDE technical assistance meetings, and site visits 
to. participating school districts. 

Four MSDE training events were observed by RBS staff. These events 
consisted of one follow-up for ML and two for STL; a three-day training 

Itute in all models for 200 participants in July 1983; and a state-wide 
conference attended by 500 educators in May 1984 at which 16 projects made 
presentations.* 



RBS staff were assisted in observing the May conference by MSDE-supported 
staff involved in another project. 
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Six technical assistance meetings were observed to determine the nature and 
extent of assistance » 'the o^crotlng opportunities » and the constraints. 
Eighteen site visits were made, two each to nine project sites. Site visits 
Included classroom observations, participant/observation of local meetings, 
and Interviews. 

In all cases comprehensive notes were taken, objectively describing what 

« 

occurred and Indicating the elapsed time. 
Interviews 

The MSDE TAs were Interviewed individually on questions relating to their 
role in SITIP, perceived successes, and recommended Improvements. 

Informal Interviews with local implementers were conducted during site 
visits and at training events. 
Questionnaires 

Three questionnaires were used: (1) Key Contact Quest lonr s (2) 
Follow-Up Feedback Form, and (3) General Survey. Each one is duocribed 
below. 

Key Conta c t Questionnaire . A key contact person was designated In each 
LEA for each model Implemented. This person was required to complete two 
versions of the questionnaire: a "pre" version in September 1983 and a "post" 
version the following May. Items on the questionnaire related to level of 
iuiplementatlon (e.g., numbers of schools, grades, teachers, classes, and 
students involved), LEA objectives for the SITIP project, and activities 
relating to those objectives. 

Follow-Up Feedback For m. LEA participants of MSDE training events were 
required to complete feedback forms. Items on the forms related to such 
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features as clarity of objectives, utility of the activities, quality of 
support and assistance from MSDE staff, and future needs. 

General Survey . A general survey was developed by RBS and completed 
by up to 20 respondents In each project from three role groups (I.e., central 
office, school administrators, teachers). Survey Items related to Implementa- 
tion, perceived impact, dissemination. Institutionalization, and technical 
assistance. 
Document Analyses 

All materials developed by MSDE for planning, training, and communica- 
tion about SITIP were reviewed by RBS. Materials (Including video tapes) 
developed by all LEAs used at follow-up meetings and state conferences were 
also reviewed. Some LEAs also provided copies of classroom materials, 
training packages, and evaluation reports. 
Student Assessment 

While the General Survey Included questions relating to perceived program 
Impact on students In terms of attitude, behavior, and achievement, local 
systems were also encouraged to collect data directly from students to 
determine their attitudes and to assess achievement. Data summaries could 
then be submitted to RBS as evidence supporting self-report claims. 

Student attitude measures . LEAs could elect to use a given questionnaire 
or one of two surveys to assess attitudes. The questionnaire (elementary and 
secondary versions) measured students* enjoyment, interest, and perceived 
learning, on a five point scale for seven simple items. The surveys were more 
complex than the questionnaire. The Learning Environment Inventory (secondary 
level) had fifteen scales measuring the following areas: competitiveness, 
satisfaction, difficulty, friction, disorganization, apathy, favoritism, 
environment, cchesiveness, speed, goal direction, cliqueness, diversity, 
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formality, and democratic. All but the last three scales relate to one or 

more models. The My Class Inventory (elementary level) measured the first 

four areas mentioned above. Copies of the measures and scoring guides were 

made available to LEAs by the TAs. Participants In the Mastery Learning 

Follow-up Workshop also attended a one-hour training session by RBS. 

Achieveme nt measures . LEAs were encouraged by MSDE to assess studer.t 

achievement. Guidelines stated: 

Achievement may be measured by the CAT or other norm-referenced 
tests, or by criterion-referenced measures. While It Is theoret- 
ically desirable to pretest and posttest students comparing 
results of SITIP students with similar populations In "regular" 
classes, this traditional design is not always possible. Alterna- 
tives include: (I) long term trend analysis comparing "posttest 
only" results with those that might be commonly expected, or (2) 
pretesting and postteatlng SITIP students using criterion 
referenced teste. Whichever method is used. It should be under- 
stood that "claims of program success" or "objective value" of 
SITIP should be made with caution. 

Specific procedures were developed for use with various kinds of 
measures: California Achievement Test, other norm-referenced testa, or 
criterion referenced tests developed by the LEA or classroom teachers. 

i 

Analysis and Reporting 

Data were analyzed for each question by model, by project, by LEA, and by 
role group. Correlational analysis were used to determine relationships among 
the variables: Instructional gain, role group support, fidelity of implemen- 
tatlon, and institutionalization at the system and school levels. Differences 
between the models, the strategies, and school types (elementary vs. 
secondary) for each variable were examined using analysis of variance. 

Data were analyzed as soon as possible after collection. Summaries were 
prepared anJ oral reports were made to the MSDE team about once a month and in 
writing for formal events. Turn around time for written reports of specific 



"critical events" was usually seven to ten days. This system of on-going 
analysis and reporting helped the MSDE team to make data-based decisions with 
regard to planned Interventions and program Improvements. 

In the fall of 1983, project coordinators received case studies and 
report sections discussing findings relevant to the model they were 
implementing. The executive summary of the report for the schoo' year 1982-83 
was distributed to local superintendents in January 1984. MSDE specified that 
local Implementers should attend to report recommendations and make data-based 
decisions to terminate or Institutionalize by June 1984. 



IV. STATE INITIATIVES AND ASSISTANCE 



In support of local implementation of four modelaijDf instructional 
improvement, the Maryland State Department of Education (MSDE) initiated 
activities in three areas: planning, training, and technical assistance. In 
order to carry out those activities effectively, organizational structures and 
mechanisms were established which built upon previous efforts. This chapter 
presents general background information for four consecutive school years, 
clascribes the organizational structures used for SITIP, reviews events for the 
three areas of activity, and presents a summary and related conclusions. 

General Background 

The first year focused on planning and orientation. Local implementation 
began in the fall of 1981. Between mid 1982 and mid 1984 (the last two school 
years of this study) additional LEAs and schools became involved, impact was 
assessed » and Indicators of Institutionalization were examined. The following 
discussion reviews relevant activities for each year. 
Year 1; 1980-81 

During the 1980-81 school year, t'.ams of educators attended orientation 
conferences conducted by the researchers/developers of the models. Then 19 
LEAs submitted plans to implement one or more of those models, received 
grants (up to $5,000 per LEA per year), and participated in further training 
and planning activities sponsored by MSDE. 
Year 2; 1981-82 

During the 1981-82 school year, nearly 700 teachers in grades K-12 used 
one or more of the models in mathematics, reading/ language arts, science, 
social studies, or other academic areas. In each of the 19 LEAs, SITIP 
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teams were formed, Including teachers, school-based administrators, and 
central office staff. These teams were assisted by MSDE technical assistants 
(TAs) — two people per model who visited local sites, facilitated problem- 
solving and networking, and conducted one or two "follow*up** training events 
attended by all teams Implementing a given model. For two models (STL and TV) 
the respective developers participated In training. 

In addition to conducting activities relating to a specific model, MSDE 
also sponsored an Instructional Leadership Conference In May 1982 focusing on 
quality instruction (addressed by Barak Rosenshine) and staff development 
(addressed by Bob Bush and Bruce Joyce) . The conference was attended by about 
500 local educators I Including SHIP team representatives. (MSDE staff 
attended a separate conference conducted by the same presenters.) 

To further reinforce the Instructional Improvement theme, MSDE 
commissioned Research for Better Schools (RBS) to write a paper synthesizing 
research on instructional improvement and planned change.* That paper was 
subsequently used as a knowledge base for several MSDE-sponsored training 
events in the 1982-83 school year. 
Year 3: 1982--83 

During the summer of 1982 » the five LEAs that Initially had not 
participated in SITIP became part of the program and received appropriate 
assistance and funds from MSDE. Training was conducted by MSDE TAs, with some 
help from STL and TV developers. Funding, assistance, and model*specif Ic 
follow-»up training events continued for all LEAs throughout the year. 



* Roberts, J.M.E., 4 Smith, S. Instructional improvement; A system-vide 
approach . 
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During the 1982-83 school yeatp nearly 1,000 teachers in 140 schools 
(about 11% of the schools In the state) were Involved In SITIP. Sixty-five 
percent of those schools were elementary, 34% secondary. Attention focused on 
program impact, which was greatest for students in AT and ML. 

In early fall 1982, an RBS evaluation report covering the 18 months 
beginning December 1980 was released, and the executive summary and sections 
relating to local implementation of specific models were distributed and 
discussed by key interest groups. Some findings influenced Subsequent 
activities* 

The spring 1983 Instructional Leadership Conference included presenta- 
tions made by each of the 19 LEAs first involved in SITIP, and focused on 
teacher effectiveness (Madeline Hunter) and planned change (Karen Louis). 
Those two presenters also addressed MSDE staff and college faculty at separate 
conferences. The RBS synthesis paper was used as advance reading for the LEA 
conference, and was the basis of several other presentations and training 
events to state and local administrative and supervisory staff. 

By June 1983, 23 LEAs were committed to a third year of SITIP implement 
tation, all with matching state funds, and all planning expansion.* 
Year 4: 1983-84 

During the 1983-84 school year, about 2,700 teachers in over 180 schools 
used one or more of the SITIP models* Attention focused on 

institutionalization, with the expectation that projects would make data-based 
decisions to terminate, institutionalize « or expand. 



* One LEA did not plan expansion and did not request MSDE funds for the 1983-84 
school year. Interested teachers/schools could continue to implement the 
SITIP model on their own. 
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In July, MSDE sponsored a three day Summer Institute to train new 
Implementers In each of the models and to assist local team planning. During 
the year other follow-up training activities were also conducted • 

In the fall, RBS evaluation reports on local Implementation were distri- 
buted and reviewed with project coordinators^ The full report was released In 
January 1984, and the findings were reviewed with key Interest groups. 

The spring 1984 Instructional Leadership Conference Included presentations 
by 16 LEAs and focused on "Elementary vs. Secondary School Effectiveness" 
(Jane Stalllngs) and "What's Good About American Education" (Harold 
Hodgklnson). Stalllngs also addressed MSDE staff at a separate conference. 

The SITIP design calls for flexible state leadership, and Involvement of 
all role groups In planning, training, and Implementation. MSDE sponsors 
planning and training events, carries out technical assistance and evaluation, 
and facilitates local implementation and dissemination. Local involvement is 
voluntary, but lip service compliance is not accepted as implementation. Local 
Investment (time, money, and commitment) is high and is influenced by the 
nature of the design, the quality of technical assistance and training, and 
the perceived value of the models implemented. In general, as SITIP gained 
visibility within the state^ central office staff (especially superintendents 
and assistant superintendents) gave more attention to the program and to the 
relevant research bases. 

Organizational Structures 
The organizational structures used for SITIP evolved over time to ensure 
appropriate participation of role groups and hierarchical levels. Multiple 
channels of communication were used, with careful attention to sending 
consistent, clear, timely messages, and to maintaining personal contacts so 
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that local educators could readily exchange Iftformatlon with MSDE. While 
appropriate attention was paid to lines of authority, cross-level or cross- 
division mechanisms were also used or developed to facilitate coordination. 
This section outlines the structures and mechanisms which did evolve. It 
refers to MSDE's internal decision-making, the placement and responsibilities' 
of MSDE staff assigned as technical assistants, and MSDE/LEA communication. 

SITIP was initj^lly designed by staff of two departmental units of MSDE, 
building on needs and successes of existing programs relating to professional 
development academies, technical assistance. Project Basic (the state 
competency-based education program), and the implementation of research-based 
processes and models. Once approved by the state superintendent, SITIP was 
reviewed by the Instructional Coordinating Council (ICC) — the state superin- 
tendent, assistant deputy superintendent, and MSDE assistant superintendents 
each responsible for a particular division/department. ICC members agreed 
that SITIP would become a jointly-sponsored program, coordinated by the 
assistant deputy superintendent (ADS), and supported by the person time of 
selected division staff with field responsibilities. These staff became the 
SITIP technical assistants (TAs) , each "expected" to spend about two days a 
month on the program. They continued their usual tasks, and, for SITIP, 
reported to the ADS. 

The SITIP TA team was chaired by the ADS and included eight TAs (two per 
model) drawn from the Divisions of Instruction; Certification and Accredita- 
tion; Instructional Jelevision; Library Services; Compensatory, Urban and 
Supplementary Programs; and the Office of Project Basic. The team met monthly 
to review progress, assist each othe*" or share materials, and to plan forth- 
coming activities. Individual members took on specific tasks most closely 
relating to their "regular" work. Most general administrative work (e.g.. 
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coordinating local plans and allocating funds) was undertaken by the two TAs 
vrho routinely reported to the ADS. Each partnership was free to determine 
what technical assistance should be offered and how work should be shared* 
Members were expected to network about SITIP within their own divisions, 
spreading successful concepts and building a general knowledge base among MSDE 
staff. This communication was not as strong as was initially planned. 

Communication between MSDE and the 24 LEAs Initially Involved the ADS and 
LEA superintendents, and that channel continued to be used for formal Informa- 
tion exchange. Subsequently, th6 local council of assistant superlntendeulb 
(that meets monthly, chaired by the ADS) became a communication channel. The 
SITIP model required involvement of cross-»hlerarchlca.' local teams, and once 
they were established MSDE TAs could contact specific teachers, school-based 
administrators, and central office staff. Usually one of the latter group 
(or, more rarely, a school administrator) was designated as the local project 
coordinator and became the key contact for LEA/MSDE SITIP communication. In a 
few cases, a project coordinator was so little Involved In SITIP that someone 
else (usually school->based) became the key contact, especially to review 
Implementation progress or needs. Important information (e.g., about the 
annual Instructional Leadership Conferences) or materials (e.g., evaluation 
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summaries) were shared In several ways through several channels (e.g., 
councils of superintendent's and assistant superintendents, mailings, TA local 
on-site visits, follow-up training events), with senior administrators 
receiving Information first, but otheiP" channels used to ensure that "desk-work 
blocks*' did not delay or prevent communication. (Even so, such blocks did 
occur at times. Indicating a need for TAs to encourage better communication 
within some LEAS.) 



Planning* 

SITIP policies and activities were planned by the TA team, with members 

taking Into account local needs and Interests. Plans were reviewed, revised 

If necessary, and approved by the ICC., Operational specifics were negotiated 

with LEA superintendents and SITIP teams. In general, the SITIP TA team took 

primary responsibility for leadership and administration of the program, with 

the ADS responsible to the ICC for maintaining quality and cost~ef l^ctlveness. 

/ 

Planning was flexible, interactive, on-going, and based on an open- 
systems appr^h. Within MSDE ami between MSDE a^d the LEAs, efforts were 
made to coordinate activities and to stlrengthen or Integrate existing programs 
with SITIP (or SITIP knowledge bases on Instruction and planned change). 
Planning was timely, made good use of resources and available expertise, and 
Invited local participation by role groups In such ways as to result in high 
commitment to the program and real (not llp-servlce) Implementation in almost 
all sites. The combination of visible siftci^ss and voluntary participation 
also facll^itated planning. 

Training 

MSDE-sponsored training activities related to SITIP during the 1983-84 
year included: (1) a Summer Institute^ (2) an Instructional Lear'.ership 
Conference, and (3) Follow-up Workshops. Each activity is described below. 

Summer Institute 

In July 1983, approximately 200 participants from 23 LEAs attended a 
three-day training session. 



For a complete discussion of planning, see Instructional Improvement in 
Maryland; Impact on Educators and Students . 



28 3 8 



On the first day, a general overview of SITIP was presented, with refer- 
ence to events to date, the research Influencing the Instructional models, and 
the processes used. SITIP findings and factors influ^enclng Instltutlonallza- 
tlon were reviewed. (Presenters Included the ADS, a TA, and an RBS 
consultant.) 

On the second day» participants formed five groups. Four consisted of 
local educators new. to SITIP who received training In Implementing a given 
model. The fifth group consisted of "experienced" SITIP participants who 
attended a session on the management of change. The model-specific sessions 
were conducted by MSDE TAs with assistance from experienced LEA staff (and the 
developer In the case of STL) . The "change"' session was conducted by the RBS 
/ consultant, l^lth assistance fromhwo SITIP project coordinators. 

Following model-specific progress reports from local projects, LEA teams 
spent the third morning planning 1983-84 Implementation. 

Evaluation forms were returned by 129 local educators (79 teachers, 35 
school-based administrators, and 15 central office staff), of whom about 53Z 
were "new" to SITIP. , 

Mean ratings of the Institute overall are shown In Table 3. "Old" 
participants had a much better understanding of Institute objectives ahead of 
time than did "new" participants. - The strongest outcome for "old" partial- 
pants was follow-^up planning » and for "new" participants was overall awareness 
of SITIP activities. Negative participant comments (28) related to Informa- 
tion overload and the need for more "how to" guidance, while positive comments 
(48) related to organization, quality of presentations, and participant 
Interactjlon. 

^^^^TJ^ session on management of ^-change. Intended for "old" participants, was 
also attended by 13 "new" participants. Mean ratings are shown In Table A. 
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Table 3 

Mean iEtatlngs of Sunoner Institute 
(Summer 1983) 



" ^ _ Respondent s 

Item — 


"Old" 
, Participants 


"New" 
Participants 


Total 


Prior to attending, I understood the purpose and 
objectives of this institute. 


4.1A 


3.2A 


3.65 


The purpose and objectives were clearly presented 
at this institute. 


A. 24 


4.32 


A. 28 


As a result of this institute I have: 








— increased my awareness of overall SITIP . 
activities and outcomes. 


3.98 


A. 33 


A. 17 


■j ^ 
— Increased my awareness of research on 
planned change and its relationship to 
the SITIP design and outcomes. 


A. 05 


3.86 


3.95 


— increased ray awareness of LEA activities 
and plans related to the selected topic. 


A. 12 


A. 21 


A. 17 


— become involved In planning for 1983-8A 
follow-up activities for the selected 
topic. 


A. 33 


A. 09 


4.20 



ej^ "Old" participants, have been implementing SITIP topic(s) for at least one year. 
"New" participants have not y«jt implemented SITIP toplc(s). 



Responses range from 1.00 (strongly disagree) to 5.00 (strongly agree). 
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Table 4 

Mean Ratings of Management of Change Session 

(Summer 1983) 





Respondents 


Item 


Old 
N-37 


New 
N-13 


Total 
N-50 


As a result of this session I have: 








* — increased my awareness of my 
individual problem solving style. 


4.19 


3.54 


4.02 


— increased my awareness of h«-'W my 
understanding of group dynamics 
can help with SITIP Implementation. 


3.89 


3.46 


3.78 


— increased my awareness of open 
systems planning for SITIP. 


3,75 


3.31 


3.63 


—increased my ability to use know- 
• ledge of individual style, group 
interaction, and systems planning 
for the management of change 
(especially SITIP). 


3.94 


3.46 


3.82 



/STte: Responses range from 1.00 (strongly disagree) to 5.00 (strongly agree). 
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"Old" partlclp.nts benefited more than "new" ones. Negative comments (five) 
Indicated that participants found the activity too general, but positive 
comments (12) stated that the presentation was helpful and practical. 

Participant ratings for the model-specific sessions ar« presenfjd in 
Table 5, and are least positive for AT, most positive for STL. Of the 95 
participants, 13 made negative comments indicating that there was insufficient 
training in ML and TV, and some repetitive activities for all models. Posi- 
tive comments were made by 50 respondents, praising the enthusiasm of the AT 
presenters, the use of group dynamics techniques in the ML sessions, the 
simulations conducted by the STL developer, and the training in coding for TV. 

Participants also s^ed their anticipated needs for the 1983-84 school 
year (see Table 6). The greatest area of need was for additional learning 
activities (through Intercounty visits, state follow-up conferences, or local 
Inservlce for oth#r teachers) . These needs were to be addressed by MSDE TAs. 

Overall, the institute was well-organized, informative, and provided 
participants with an opportunity to Interact with each other. 

Instructional Leadership Conference* 

In May 1984, approximately 500 participants attended the fourth annual 
Instructional Leadership Conference.** Most participants were local 
educators. The audience also Included MSDE etaff, state and local board 
members, educators from private schools, and representatives from institutes 
of higher education (IHE). Topics and presenters included: 
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* In addition to the major conf trance designed primarily for local educators t 
another was held the following day for MSDE staff. The second conference 
consisted of a presentation by Jane Stalllngs and opportunities for SEA 
staff to Interact with her* 

Previous conferences designed to Improve Instructlont teacher effective- 
nesSt and planned change were held In 1981 (featuring Bloomt Goodt Slaving 
and Helms) t In 1982 (featuring Rosen6hlne» Busht and Joyce) t and in 1983 
(featuring Louis and Huntisi*), ^2 
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Table 5 



Mean Ratings of Model Specific Sessions 
(Summer 1983) 





Model Ratings 


Item 


AT 
N-21 


ML 
N-28 


STL 
N-33 


TV 
N=13 


Overall 
N-95 


As a result of the training: 












— I have increased the knowledge 
and skills I need to implement 
the model. 


3.81 


4.10 


4.52 


4.46 


4.23 


—I feel more confident and 
enthusiastic about iny involve- 
ment. 


3.75 


4.14 


4.36 


4.38 


4.16 


—I feel as if I belong to the 
SITIP "team." 


3.80 


4.00 


4.41 


4.23 


4.13 



Note ; Responses range from 1.00 (strongly disagree) to 5.00 (strongly agree). 
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Table 6 



Needs of Institute Participants Implementing Each Model 

(Summer 1983) 



Needs 

. 


AT 
N-26 


ML 
N-26 


STL 
N-37 


TV 
N-13 


Overall 
N-103 


Continue general support and assistance 


3 


3 






D 


Have follov--up sessions 


I 


3 


A 


1 


9 


Help conduct LEA Inservlce 


5 


3 


4 


3 


15 


Arrange Intercounty visits 


6 


3 


8 


1 


18 


Provide model specific "Information" 
or materials 


1 


7 


5 


2 


15 


Help observe/critique classroom 
Implementat Ion 


• 

2 








Z 


Provide evaluation tools/techniques 


3 


1 






A 


Provide RBS reports-feedback 




1 






1 


Arrange for (or fund) teacher 
release time 


1 


2 


6 


2 


11 


Other 


2 






A 


3 


Not sure yet 


2 


3 


10 




19 
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• similarities and differences In effective elementary and secondary 
schools — Jane Stalllng8» one hour» all participants 

• Communicating vhat*8 good about American education — Harold 
IIodgklnson» one hour» all participants 

• Role group responsibilities and curriculum subjects addressed In 
successful school Improvement projects — LEA teams and RBS staff » nine 
concurrent sessions of A5 minutes 

t Factors facilitating expansion and Institutionalization of school 
Improvement project n — LEA teams and RBS staffs nine concurrent 
sessions of 45 minutes* 

The selected topics* Involvement of researchers » participation by cross- 
hierarchical local teams (Including LEA superintendents) » and the overall' 
"message" of encouragement to link research and practice^ all reflected the 
SITIP philosophy familiar to SITIP lmplementers» but not necessarily under- 
stood by non«SITIP participants. 

The two keynote speakers referred to results of major research projects » 
Stalllngs drawing primarily on classroom effectiveness studies » and Hodgklnson 
referring to Headstart and Follow- through studies and demographic data* 
Stalllngs was teacher-centered » while Hodgklnson had a broader perspective. 
In addition to those two presentations — one first thing In the morning, the 
other Immediately after lunch — there were two rounds of small group 
concurrent sessions. In each round » one session was conducted by RBS staff 
who reviewed findings of the SITIP study. Al80» LEAs In each round made 
presentations describing particular aspects of their Implementation of SITIP 
models. (LEAs were selected based on their relative success In a topical 
area» as for Instance^ effective use of cross-hierarchical teams » or quality 
development of curriculum units.) Most small group participants received 
coplej of relevant handouts. 
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Completed evaluation forma were returned by 230 participants (71% LEA» 
15% MSDE, 6% private/parochial schools, and 3% IHE). Forty eight percent of 
the participants had no prior Involvement with SITIP. 

Participant ratings of keynote and small group presentations an4 of the 
overall conference are presented in Tables 7, 8, 9. VThlle both Stalllngs and 
Hodgklnson received better than average ratings » Stalllngs was less popular » 
probably because she presented information already known to SITIP participants 
and she was difficult to see and iiear. By comparison » Hodgklnson had an 
enthusiastic^ energetic style and could be clearly heard and seen. 

Responses to the concurrent sessions were grouped according to the focus 
of the presentation*. In both rounds RBS* presentations were the least 
popular (possibly because of the research focus). Overall » the expansion 
presentations were most popular. All small group ratings were above average. 

Overall conference ratings were good. Negative comments (26) » of which 
54% were from SITIP lmplementers> indicated that SITIP Implementers did not 
like the facilitators or were looking for something new» and non-SITIP 
Implementers were disappointed because their needs were not addressed (e,g.» 
specific Information on SITIP models » advance reading materials » more 
discussion tlme» non-research presenters). Positive comments (16) » of which 
56% were from SITIP l]iq)lementers» were simply generally appreciative. 

While the conference was a success » there was a problem caused by 
audience diversity In relative sophistication of knowledge on research and Its 
application for classroom effectiveness. Results suggested that If a fifth 



* In Round 1» elementary programs were discussed by Dorchester » Montgomery » 
and Wicomico; secondary programs by Anne Arundel » Calvert » Carroll » Queen 
Anne*s» and St. Mary*s; general role group responsibilities by RBS. In 
Round 2» program expansion was discussed by Allegany» Cecll» Garrett /and 
Howard; Instltutlonallsatlon by Charles^ Somerset » and Worcester; both 
dimensions by Baltimore County and by RBS. 
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Table 7 

Mc I Ratings of the Keynote PrtAcntations (May 1984) 



Rating 

Item 


Stallinga 


Hodgkinson 


Total 


Relevancy of object ivta 


3.73 


4.68 


4.17 


Accomplishment of objectives 


3.44 


4. 68 


4.02 


Overall quality of the presentation 


3.36 


4.76 


4.01 


Total 


3.51 


4.71 


4,07 



Respondents rated the presentations on a scale from I. 00 (poor) 
to 5.00 (excellent). 



Table 8 

Mean Ratings of Small-Group Preaentations^ (May 1984) 





Round I 
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H 










H 


Relevancy of objectives 


4.05 


3.99 


3.24 


3.91 


4.23 


4.13 


3.73 


3.91 


4.08 


Accomplishment of objectives 


4.02 


3.97 


3.29 


3.90 


4.19 


4.08 


3.47 


3.87 


4.02 


O'.'erall quality of the presen|ition 


4.01 


3.es 


3.25 


3.83 


4.19 


4.08 


3.47 


3.87 


4.02 


Total ^ 


4.03 


3.94 


3.25 


3.BB 


4.20 


4.10 


3.56 


3.88 


4.04 



Respondents rated the presentat iona on a scale from 1. 00 (poor) to 5*00 (excellent) 



Table 9 

Mean Ratings of the Overall Conference (May l<l84) 



Item 


Rati g 


Conference design (format, use 
of timet range of activities) 


4.12 


Conference facilities 


4.47 


overall quality of conference 


4.10 


Value in terms of use of 
partic Ipants* time 


3.93 


Total 


4.16 



Respondents rated the conference on a scele from 
1.00 (poor) to 5.00 (excellent). 
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annual conferenca la hald It should ba designed to take Into account partici- 
pants' "prior learning, " and their attitudes to the philosophy and concepts 
Influencing SITIP-related activities. 



Follow-up Workshops 

During the 1983-84 school year, four follow-up workshops were conducted 
by MSDE TAs. Each was designed for local educators Implementing a given 
model, and Included Information, materials, and activities requested by local 
participants. Two workshops were conducted for STL, one for ML, and one for 
TV.* 

Mastery Learning follow-up workshop . The ML follow-up workshop was 



conducted In Howard County In November 1983. It was attended by about 40 
local educators representing all eight LEAs Implementing the model, three MSDE 
staff, a local consultant, and an RBS participant observer. In preparation 
for the event, LEA teams completed a needs assessment survey and MSDE TAs 
developed the following agenda to address those needs: a review of current ML 
literature; and two rounds of concurrent sessions, with "Enrichment" and 
"Keeping the Project Alive" each conducted twice, and "Ability Grouping" and 
"Affective Assessment" each conducted once. Following lunch, participants 
reviewed the RBS evaluation report (using the STL Jigsaw model for peer and 
small group discussion), explored networking Ideas, and attended a presenta- 
tion on the use of microcomputers. 

• The literature review was presented by an MSDE TA (CUSP). Articles 
were displayed, and "tagged" so that participants could request 
copies. This brief session was well received and many participants 
requested (and were given) copies of the articles. 



* There was no workshop for AT because the "lead" MSDE TA resigned In early 
1984 and the other AT TA had too great a work load to conduct a follow-up 



worka''op. 
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"Keeping the Project Allve/Inaervlce" was presented twice by a former 
TA (C&A) who referred to such Ideas as: three essentials — selling 
the philosophy, defining the vocabulary, training In the program; 
change elements — total Involvement, group agenda. Incentives, Bruce 
Joyce Inservlce model (with emphasis on coaching). During the 
sesalon, participants talked In small groups, shared Ideas with the 
large group, and questioned the presenter. The presenter suggested 
that It was Important to make a purposeful decision to keep the 
project alive or to kill It, and to act on that decision. Both 
sessions were well attended and participants were actively engaged. 
They liked the Joyce model and seemed responsive to the presenter's 
style. However, much of the discussion seemed to focus on "what was 
wrong" rather than on what might be done to maintain or. expand the 
program. 

The Enrichment session was conducted twice by the new TA (ITV) who 
used an overhead and transparencies, and had a few participants first 
focus on criteria for enrichment activities, then discuss Ideas with 
each other, and engage In question and answer discussion with her. 
The small group discussions Included reference to higher cognitive 
level learning, and participants shared classroom materials with each 
other. The session might have been better attended or more successful 
If purposeful attempts had been made to build support groups focusing 
on specific subject areas and grade levels, and If more sample 
materials had been available. 

The single session on Ability Grouping was conducted by the CUSP TA 
who presented a summary of. relevant research, had participants work In 
small groups to determine desirable .'pUcy in light of that research 
and of local conditions, and then reconvened the whole group to review 
Ideas. Projected transparencies Included a list of the elements of 
hetergeneous vs. homogeneous grouping. A handout presented six key 
questions and responses. Small group discussion was animated and 
thoughtful. Following an exchange of ideas, the presenter stated that 
he would supporti^^terogeneous grouping for the school, but regroup 
for basic sklllsVlUbJects, being careful not to "track" a child 
permanently. /\ 

The session on Affective Assessment was conducted once by the RBS 
participant -observer who reviewed the two measures used to measure 
student affect in the 198?-83 school year: The Leamjn^ Knvirnnmfint 
|nv«intni-v (LEI) and My niaaa Tnvanfnrv (MCI). Participants reviewed 
the definitions of th« scales, engaged in discussion of the implica- 
tions, and were told how to score and analyze results if they used the 
measures. In small groups they discussed which scales they might use, 
and why, and then shared ideas with the total group. By the end of 
the session participants appeared to have a good understanding of the 
inventories and their use. » 

In order to review the evaluation report on ML, participants worked in 
"expert" and "learning" groups, using the STL Jigsaw model. Everyone 
gained an overall understanding of the findings. 
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• To facilitate networking , participants In small groups responded to a 
list of seven questions to summarize collaborative activities In which 
they had been Involved, and to suggest Ideas for future networking and 
how best MSDE TAs might support their efforts. Some of those Ideas 
were reviewed In the large group, and TAs promised to follow through 
on some of the Ideas. 

• Computer hardware and software were discussed by Jack Cole (a non-ML 
local educator), who provided handouts and review materials, described 
such programs as Vislcalc, Wordstar, and Bank Street Writer, and 
suggested "do's and don't's" In materials selection and keeping 
up-to-date on educational technology. Participants were attentive 
during the presentation. 

Participants' responses on feedback forms Indicated that about 40% of 
them became Involved In ML In the summer of 1SL83. Ratings of the overall 
workshop and MSDE support vera good. In general, the oomlng's small group 
sessions were more popular than the afternoon sessions. The least popular 
session was "Enrichment," and the most popular "Keeping ML Alive." Responses 
— during sessions and on feedback forms — suggested that some participants ^ 
had not been inVolved In the needs assessment survey, and/or had not been 
briefed about the nature and purpose of the follow-up. Needs and concerns 
expressed — to be addressed by TAs during the school year related to time, 
Inservice materials, and others kinds of support. 

It was apparent that the two TAs spent a great deal of thought and energy 
on designing activities to meet stated needs. The follow-up seemed (on the 
whole) to be designed to provide participants with a sound knowledge base 
which they could apply in their LEAs. That purpose was achieved to some 
extent by all sessiona, and to a large extent in such sessions as the one on 
ability grouping. In subsequent months, TAs responded to local needs during 
site visits and compiled descriptions of ML curriculum units and distributed 
them to local teams. 

In the spring of 1984, the former TA (from C&A) conducted a national 
workshop with James Block (who conducted the original training in Maryland in 
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1981) • Some LEAs sent representatives to that workshop (at a reduced cost) 
since no spring follow-up workshop was conducted. 

Student Team Learning follow*up workshops . In preparation for the 

workshops » MSDE TAs were l)(^fluenced i>y suggestions made by participants at 

/ 

previous training events And by their local observations. (Classroom visits 
were popular » and host fifchools were stimulated to do well.) 

Two STL follow-'up Workshops were conducted » one in November 1983 in 

Calvert County » and oiie In May 1984 In Dorchester and Queen Anne*s counties. 

/ 

Each was attended by/ two MSDE TAs» an RBS observer » and over 40 local 

educators. The secbnd was also attended by an STL developer from John 

/ 

Hopkins. At both events, participants spent about half their time observing 

/ 

/ 

classroom use of/ STL models. 

/ 

The flrst^/follow-up workshop Included the following activities: 

• Partl<^lpants observed classrooms, in two elementary schools and a 
middie school, seeing all three models (STAD, TGT, and Jigsaw) in 
various subjects and grade levels. 

• Project reports were given by each LEA team: training and coaching 
within and between counties did occur, encouraging program expansion; 
and the "key teacher" system appeared to be gaining popularity.* 

• A presentation on TV (time-on-task and content alignment) was given by 
two Calvert teachers, who described TV procedures and implementation 
activities in the county. 

Responses to feedback forms indicated that about 38Z of the participants 

had become Involved In STL Just since July. Overall » the workshop was rated 

as very good» with classroom visits and the TV presentation the most popular 



* Key teacher (s) are knowledgeable advocates of a given model. They train and 
coach other teachets» coordinate activities within a schools and act as 
communication liaisons with other schools. They function as Instructional 
leaders » with logistical support from administrators, (See Teachers as 
Instructional Leaders for a more complete discussion.) 
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activities. Needs and concerns expressed related to time for curriculum 

development and training. When asked for suggestions for the spring follov-up 

workshop^ participants urged: sharing curriculum materials designed for STL» 

classroom observations » and strateglzlng to Improve or expand Implementation. 

The second workshop Included the following activities: 

e Participants observed classrooms in an elementary school In Dorchester 
County and a middle school in Queen Anne's» In various grades and 
subject areas » and all three STL models. 

e Project reports were given by Dorchester and Queen Anne's staff. 

e The Johns Hopkins representative encouraged participants to continue 
Implementation after state funding ended » and the MSDE TA promised to 
compile and disseminate descriptions of STL curriculum and training 
models. ' ^ 

Responses on feedback forms were very posit ive» with the most popular 

activity being classroom observations. Needs and concerns expressed related 

to time for curriculum development and opportunity to share materials across 

LEAs. 

Teaching Variables follow-up workshop . In preparation for the workshop, 
MSDE TAs were Influenced by the findings of the RBS report (for the 1982-83 
school year) and their observations during site visits » Both sources of 
information indicated tensions and misunderstandings between role groups in 
several counties, and lack of attention to the "content" variable. 

The TV workshop was held In Talbot bounty In February 1984 and was 
attended by two MSDE TAs, an RBS observer, and 33 local educators representing 
all six Implementing LEAs.* The morning activities focused on project 
progress reports, and the afternoon on school climate. 



' Two LEAs were each represented by only one role group: teachers from 
Frederick, and a central office supervisor from Kent. 
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• Progress reports Indicated that: (i) all LEAs believed they had high 
student engagement rates (tlme-on-taak) , (2) two LEAs had teacher 
controlled programs » one was controlled by administrators » and the 
others ve^e mixed » (3) one LEA Implemented "content" as well as 
"tlme/^ and another Integrated Active Teaching with TV. No LFA 
discussed strateglzlng for Instructional Improvement. 

If 

• The MSDE TA encouraged attention to the '^content" variable, and 
emphasized the Importance of support » cooperation » and trust among 
role groups. 

• ^School climate was reviewed by Calvert educators who emphasized 

principal support In making sure that teachers enjoy working at the 
schools and team work to share expertise and responsibility. 
^ '> 

• A series of activities Involved participants In discussion of school 
climate: (1) In a '^flshbowl" process » a cross-hierarchical group 
discussed Impeding factors and helpful actions relating to school 
climate; (2) three small groups ( teachers » school-based adminis- 
trators » central office staff) discussed role grdup Implications; and 
(3) LEA t^ams discussed county-specific problems and activities. 

Responses to feedback forms were positive » with the afternoon fnlall group 
sessions most popular. Needs expressed related to funding and MSDE support In 
addressing county specific needs. 

Summary . Follow-up workshops did not focus on training specific to" 
Implementation of the models » but were designed to meet needs expressed by 
participants or Identified through evaluation and on-site observation. ML 
participants gained knowledge In areas that ^u Id possibly Improve classrooin 
Implementation (e.g.» groupings enrichment^ evaluation) or program management 
(e.g.» keeping ML allve» using computers). STL participants gained knowledge 
that could Improve classroom Implemehtatlon. TV participants gained knowledge 
that could Improve program management. In all cases » there was opportunity 
for participants to learn about each other's projects and to share Ideas. 
Independent networking was most encouraged for STL. 

Participant ratings on four criteria are presented In Table 10. (A five 
point scale was used with 5 as most positive » 3 as average » and 1 as least 
positive.) Overall ratings were good» with STL ratings highest (possibly 
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Table 10 



Mean Ratings of Follow-Up Workshops 
(1983-84 School Year) 





Ratings 


by Model 




ML 
N-36 


STL 
N-37 


TV 
N-28 


Total 
N-101 


Prior understanding of objectives 


3.69 


4.46 


4.00 


4.06 


Activities met objectives 


3.89 


4.32 


4.15 


4.12 


Activities met participant needs 


3.68 


4 


4.04 


3.90 


Quality of MSDE support 


4.00 


4.43 


4.07 


4.18 



Ratings are on a five point scale » where 5 Is most positive. 
STL ratings Include both workshops. > 

reflecting the positive affect created at STL events) » and HL lowest (possibly 

Influenced by the more cognitive task orientation of ML events) . The rating 

relating to needs being met was Influenced In part by the fact that about 40% 

of participants were relatively new to SITIP while others had been Involved 

yfilnce 1981. In general* follow-up workshops were designed to meet participant 

needs, Included appropriate activities » and provided evidence of the TAs* 

professionalism and hard work (especially In ML» since that workshop was the 

most complex) . 

Over the three school years of SITiP Implementation > 15 follow-up 
conferences/workshops were conducted by TAs (six In Year 1» five In Year 2, 
and four In Year 3). Content and participant activities for each model for 
each year are summarized in Table 11. The most common activities were 
presentations J>y LEAs about their projects, and model-specific presentations 
by various individuals and groups. Planning and planned change* addressed at 
the Joint workshop (AT» STL» TV) and workshops for ML and STL> was the next 
most common topic. Other topics addressed by at least two groups of 
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Table 11 



Teaching/Learning Activities of Follow-Up Workshops ' 



Content and 


nooeis 


Across the Three Years* 




Process 










Joint 




AT""* 


ML 


STL 


TV** 


AT QTT TV 
/\X f d lb f 1 V 




1 2 


1 2 3 


1 2 3 


1 3 


2 


Model-Specific 












Presentation 












-by developer 






XXX 


X 




-by out of state expert 




X 




X 




-by TA 




X 








-by LEAs 


X X 


X 


XXX 


X X 


A 


Model-Related 












Other SITIP model 




X 


X 






Instructional research 


X X 


X 








Implementation 












Planning/planned change 




X 


X 




X 


Assisting new sites 




X 








Using computers 




X 








Staff development 




X X 








Improving climate 








X 




Networking/ teaming 




X X 








Dissemination 




X 








Student assessmer t 












& progtam evaluation 


X 


X X 






X 


Participant Activities 












Large group 


X X 


XXX 


XXX 


X X 


X 


Small groups 


X X 


XXX 


XXX 


X 


X 


Class observation 






XXX 






Video presentation 


X X 


X 


X X 







* Year 1-July 1981-June 1983; Year 2-July 1982-June 1983; Year 3-July 
1983-Jane 1984. 



** No model-specific workshops were conducted for AT In Y^ar 3, or for TV 
In Year 2. 
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participants Included other SITIP models • related Instructional research i and 
evaluation. Both large and small group sessions were almost always conducted, 
and all models except TV used video tapes. Classroom observation occurred 
only for STL. The greatest emphasis on model specific training was for TV and 
STL; the greatest diversity was for ML; for AT, Intensive exploration of 
Instructional research was the major focus. 

The original objective for the workshops was to support local Implementa- 
tion by Increasing participants* expertise and confidence In the modelCs) 
Implemented. TAs also wanted to facilitate networking among LEAst and to 
expand participants' knowledge of teaching/learning and dimensions of Imple- 
mentation. All three objectives were addressed for all models to some extent, 
with affective objectives best achieved by STL, and cognitive objectives best 
achieved by AT and ML. The cllent-responslve TA philosophy led to programs 
designed to meet expressed participant needs i Each TA ( * dyad) was also 
Influenced by personal preferences or biases (e.g., AT involvement of related 



research, ML emphasis on staff development). 

One Important area of need that could have been addressed more 
effectively (and might be In 1984-85) related to the "rich fidelity'' of the 
models. That is, once teachers could implement a model for a given course or 
instructional unit, they might justly claim fidelity of implementation since 
they followed all the steps, yet the application might be relatively super- 
ficial. Training to improve "richness" might have Included: (1) for AT, 
attention to piecing together component skills to build concepts, and 
attention to exemplary classroom management (e.g., by using videotapes made in 
some LEAs); (2) for ML, attention to alternative ways of teaching (rather than 
simply reteaching) for correctives; (3) for STL, attention to increasing 
teachers' understanding of small groups. Increasing curriculum development 
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skills, and attention to Integrating STL etrategles and materials Into 
district curriculum; and (4) for TV, attention to the "content" variable, and 
to atrategiziyig for instructional Improvement (rather than focusing on data 
collection and coding). For all models, workshops might have benefited from 
exploring ways in which one might reinforce another. (For Instance, one LEA 
Integrated AT and TV, and another used STL with some ML classes.) Finally, 
since workshops were attended by all role groups, often the sami individuals 
over a two or three year period, activities should have built upon those facts 
(as did ML and — during Year 3 — TV). 

Summary 

Training was designed for cross-hierarchical teams, supported implementa- 
tion of the SHIP models. Included information and activities to reinforce 
content and process, took into account participant needs and Interests, 
Involved local teams as presenters. Involved outside consultants as presenters 
(carefully coached by MSDE TAs) , and was provided on the understanding that 
MSDE would provide assistance for LiJAs wishing to follow ideas through with a 
larger number of local educators. The various kinds of training events 
reinforced each other. One recent problem vas the discrepancy between 
"veteran" SITIP implementers and those who knew nothing about the program. 
The varying levels of knowledge were not always addressed to participants' 
satisfaction. This suggests that MSDE needs to explore alternative designs 
for future events. 

Over the three years, seven statewide conferences were held, each 
attended by as many as 500 LEA participants. Related conferences were held 
for MSDE and IHE staff. Presenters considered to be national experts on 
instructional improvement, staff development, and planned change included: 
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David Berliner^ James Blocks Ben Blooi&»* Robert Bu8h»* Barbara Clements* 
Michael Cohenr Tom Good»* David Helms et al.»* Harold Hodgkinson** Madeline 
Hunter Bruce Joyce » Susan Loucks» Karen Louis Jane Roberts* Barak 
RoBenshlne** Robert Slavln et al.»* and Jane Stalllngs.* Also» 15 follow-up 
workshops were conducted » attended by as many as 45 local educators 
implementing a given model. Many on-site local training sessions were 
conducted by TAs» or by LEA coordinators with TA help.** Within schools and 
school systems trouble-shootings program development^ and coaching occurred^ 
often with TA help of some kind. In Year 2» many interrelated training and 
knowledge building activities strongly reinforced the ''message** of applying 
research for Instructional improvement. This was clearly evident in Year 3 
when some LEAs» regardless of the model adopted » addressed such general 
concepts as tlme*on-task and curriculum alignment » weaving them into training 
and coaching. Also» processes of planned change were strengthened » and 
central office staff became more ^ware of the impact of their involvement. 

Overalls participant evaluation^ process observers* nctes^ and subsequent 
locals states and college action provided strong evidence of the value of the 
SITIP-related training sponsored by MSDE. 



* Only asterisked presenters addressed the total audience at a statewide 
conference. Others made presentations to MSDEs IHEs or smaller groups of 
SITIP implementers. 

** Assistance and Information exchange from developers was invited by lAs. 
STL developers responded s participated in many MSDE training event Ss 
volunteered information on new SFL featuress ends in turns initiated 
requests for information. TV developers conducted training at three LEAs 
and at two follow-ups s and responded to TA requests far Information. ML 
developers maintained frequent information exchange with TAs; some local 
implementers attended national Ml training events. AT developers did not 
respond to LEA communications and maintained minimal contact with TAs. 
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Technical Assistance 
As stated previously, assistance to LEAs was provided by an eight-person 
team under the leadership of the ADS. The team carried out planning and 
training activities described earlier In this chapter and also worked In dyads 
to provide model-specific assistance to local Implementers. This section 
describes the technical assistance (TA) system, the roles and responsibili- 
ties of the TAs, their Impact, and the evolution of the role. 

The TA System 

The ADS provided leadership, allocated resources, and encouraged volun- 
tary acceptance of tasks to be done. He chaired six meetings (usually lasting 
one or two hours) during the year, and coordinated activities across models. 
While he encouraged each TA dyad to be program-oriented and autonomous, he 
established expectations for major activities such as collection of local 
plans. 

The TAs were drawn from various MSDE divisions. At the beginning o*" the 
school year there were: two administrators from Project Basic; two staff and 
a staff development consultant from the Certification and Accreditation 
Division (C&A); two Instructional staff — both mathematics specialists — 
from the Division of Instruction; and three program specialists, one each from 
the Divisions of Library Development and Services (LDS), Compensatory, Urban 
and Supplementary Programs (CUSP), and Instructional Television (ITV) . 

Two TAs were assigned for each model. For the 1983-84 year the following 
changes occurred. 

• For AT, the first TA (a mathematics specialist) stayed with the 
project until February 1984, when he resigned from MSDE. A second 
mathematics specialist Joined the TA group in June 1983 and stayed 
with the project through 1984. 
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• For ML, the CUSP program specialist TA continued to take the lead. 
The staff development branch chief handed over responsibility to a new 
TA (In Instructional Television) in June 1983. In June 1984, the ITV 
TA handed over rtsponslblllty to a different ITV staff member. 

• For STL, two TAs remained in their positions, one from LDS, and one 
from Project Basic* The former left SITIP in June 1984 and was not 
replaced. The latter maintained her leadership position throughout. 

e For TV, both TAs continued with SITIP, proposing to share work 

equally, but finding that the administrator TA Invested somewhat more 
effort than did the staff development TA. 

The TA system was loosely-coupled, decentralized, program-oriented, and 
made up of highly-autonomous members held accountable for maintaining produc- 
tive working relationships with LEAs. As long as local feedback to the ADS 
(e.g., from LEA assistant superintendents, ICC members, or RBS evaluators) was 
positive, each TA was free to use his/her own Judgment. 

When a TA needed assistance, he/she asked for and received help from 
another team member, including the ADS, and members were well aware of each 
others' strengths. Thus, the TA system provided support for its members, and 
coordinated administrative and logistical planning, communication, and 
resource allocation. While TAs worked as a team to plan Leadership Confer- 
ences and the 1983 Summer Institutes, and to ensure consistent and appropriate 
communication to LEAs about SITIP (e.g., planning and evaluation requirements, 
resource allocations), they did not work together for delivery of assistance 
to LEAs. In general, as a team and within each dyad, TAs made appropriate 
arrangements to get the work done, usually without Interpersonal conflict and 
without things "falling through the cracks." Quality and quantity of work 
done were Influenced by TA perceptions, by the level of effort Invested in 
specific tasks, and by organizational arrangements within each dyad. 
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Responsibilities, roles, rewards, challenges, and tasks, are discussed 
below for the team as a whole, with some references to model dyads. 

Responsibilities . Vfhile all TAs agreed that their responsibility in the 
third year of implementation was to help local educators assume primary 
responsibility for the future of their projects, each dyad defined that 
responsibility slightly differently. 

Roles . One TA believed the LEAs did not need MSDE assistance during the 
third year of implementation. The remaining TAs believed that they served a 
useful purpose and that the role included: coordinating networking among LEAs 
using the same model; visiting the various sif.es to acknowledge successful use 
and to help plan and problem solve; maintaining interest among "veterans;" 
training "new" implementers (mostly at the July 1983 institute); and 
encouraging local ownership and independence. 

Rewards . The initial excitement of SITIP waned, and the various rewards 
perceived by TAs became more integrated into their regular roles. Of the 
seven TAs who served the full year, one was pleased to see LEAs "standing on 
their own," three enjoyed seeing students and teachers doing well, four 
enjoyed the personal contact with local educators involved in instructional 
success, and five had gained knowledge and skill which helped them in their 
regular roles. One SITIP TA found no rewards in the role since It was 
perceived as very different from the regularly assigned role.* 

* Over the three years, 13 individuals held the TA role, including two who 
were "delegated" to the assignment. Four resigned from MSDE (one retiring, 
the other three accepting more challenging and rewarding positions In other 
organizations). Two received promotions. Two (including one who resigned) 
became involved in national-level activities directly related to SITIP 
responsibilities. One was assigned to provide leadership to a new SITIP- 
like MSDE project. These kinds of professional "rewards" are fairly common 
for those involved in TA activities. 
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Challenges « Regardless of the perceived rewards, all TAs experienced 
some common challenges relating to conflicting demands 9 communication/ learnings 
and relationships. 

The challenge of confllctln^j; demands — between SITIP and regularly 
assigned responsibilities — continued to concern those TAs whose regular role 
was very different from their SITIP assignments. While all TAs officially had 
15Z of their time allocated to SITIP each year (about 33 days), in practice 
their regular supervisors expected priority to be given to regular tasks.* 
When faced with a direct choice 9 three of the seven TAs were more likely to 
*Mrop** the SITIP task. Two other TAs designed their work to integrate SITIP 
and regular tasks (and travel). Two TAs did not experience the problem to any 
great extent, partly because in their regular roles they reported to the ADS, 
who treated the 15% allocation as a reality. Individual perceptions of role 
conflict were supported by other evidence. While such conflict was not 
resolved, its impact was reduced when the TAs redesigned tasks and schedules, 
or negotiated a reduced work load with the partner for a given model. 

The challenge of communication or learning was experienced by all TAs and 
was strongest for newly assigned staff. They had to learn about a model, 
local projects, and SITIP processes. They also had to learn the norms of the 
TA system and how to carry out the new role. They had to learn "the right 
questions to ask" and find out the extent to which (in SITIP) they should or 
could initiate. While the ADS and partners in dyads helped new TAs by 
offering reading mater-'al, formal communication (since it was between 
different divisions, and, in one case, between buildings 15 miles apart) did 



* The actual amount of time spent on SITIP ranged from 17 days (by the person 
perceiving the greatest role conflict) to 35 days (by one of the two people 
also perceiving high role conflict). 
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not always provide the kind of information needed. Informal communication was 
greatly influenced by staff/office proximity (opportunity to interact) and 
personal relationships. Some individual TAs did not become part of that 
informal system. 

The challenge of relationships related to differences between state and 
local expectations and actions. For instance, four LEAs did not allocate 
SITIP funds to STL projects involved in prior years, investing in other models 
or programs instead, and hoping that STL activities would institutionalize 
themselves. The STL TAs believed they should continue to provide assistance 
to those projects, involve them in networking activities, and try to maintain 
program enthusiasm. However, communication was difficult, and LEA central 
office staff did not facilitate MSDE TA efforts. For other projects, 
reiatlonship problems arose when LEA staff expected greater assistance and 
responsibility for program decisions than MSDE TAs were willing to give. For 
instance, IocmI staff reassignments sometimes resulted in loss of the local 
SITIP advocate/ ttainer, and the LEA team then expected the MSDE to assume that 
role, but the TA (while willing to assist) expected local educators to take 
primary responsibility since the program was in its third year. In two cases 
where this problem occurred, fidelity and scope of implementation suffered 
even though TA help was very high in one case. In other cases, while the TAs 
expected to be "letting go," LEA staff expected them to make decisions about 
local issues. Since the TA role did change over the three years, 
clarification of expectations (overall and for each year) would probably have, 
been useful. 

In general, the challenges of the experien(ife did reduce the TAs* level of 
effort and enthusiasm, but did not out-weigh the reward^ of the TA role, nor 
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did they reduce the quality of the wbrk done (partly because p&rtners 



compensated for each other) . 

Tasks . The ten task areas originally Identified were also addressed 
during the 1983<-84 school year, although time allocations changed, and some 
areas were slightly redefined. (See Table 12 for an analysis of time 
invested.) burlng the twelve months ending June 1984, TAs spent about 202 
days on SITIP, addressing the task areas of : (1) administration and budget 
(9X), (2) planning (loZ)', (^) knowledge building (6Z) , (4) materials develop- 
ment or Identification (3X) , (5) training (22Z), (6) general support (6Z), (7) 
visiting sites (24X), (8) evaluation (4Z), (9) communication (8%), and (10) 
dissemination (8%).* • 

During the 1983-84 year these task areas were characterized as follows. 

• Administration and budget consisted primarily of distributing and 
collecting forms from LEAs relating to funding and to local participa- 
tion in SITIP training events. It also Included record keeping of 
various kinds. No TA spent more than four days on this task area. 
About: twice as much time was spent by the two TAs whose "regular" role 
as assistant to the ADS made it simpler for them to do the work than 
for the others. 

• Planning Included activity among the TAs during monthly meetings to 
design such events as the Instructional Leadership Conferences, or 
their follow-up training workshops. No TA spent more than four days 
on planning. 

e Knowledge building consisted primarily of keeping up to date on the 
research on planned change and one or more SITIP models. TAs each 
spent one to two days on this activity. 

• Materials development/identification Work was minimal, with two days 
invested for each model except TV. This task consisted of compiling 
catalogues of locally developed materials, and/or developing materials 
such as video tapes for LEA use in training. 



* TAs were assisted in these tasks by RBS staff, who spent most of their time 
on evaluation (including site visits) and dissemination of findings (often 
through training). The total time invested by RBS (June 1983-June 1984) was 
approximately 260 days (1.20 full time equivalent). 
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Table 12 

TA Time Invested by Task: 1983-8A 
(as percentages of 202 days) 




• Training Included participating In and conducting sessions at the 1983 
Summer Institute, at follow-up workshops , and at LEA workshops. Each 
TA spent between four and eight days on this activity, with about 
equal time Invested for each model. 

• General support Included over-the-phone assistance, networking sites 
with common needs or Interests, and sharing Information. Each TA 
spent one or two days on this activity with the exception of one ML TA 
(ITV) who did not provide general support. 

• Visiting sites was performed by all TAs, each spending between three 
and eleven days. The objective was for each LEA to be visited twice, 
with more visits for "new" sites. Specific purposes varied, and 
included: 

- monitoring fidelity of Implementation 

- recognizing/acknowledging teachers* accomplishments to 
resolving problems 

- participating In cross*hlerarchlcal team meetings to 
review progress and determine next steps 

- working with project coordinators to design training, 
develop Implementation plans, or determine how to overcome 
barriers 

- building working relationships and mutual trust at all 
hierarchical levels so the model could be Implemented. 

• Evaluation was conducted by RBS, but TAs determined criteria for the 
design. Informed LEAs of their responsibilities, clarified RBS* guide- 
lines, helped LEAs develop evaluation plans, distributed and collected 
surveys and reports, and arranged for RBS to visit pilot sites. All 
but two TAs contributed to evaluation tasks, each Investing one or two 
days. 

e Dissemination took up to four days of a TA's SITIP time and Included 
making SITIP-related presentations to key interest groups such as the 
ICC and profassional associations, or at training academies; 
responding to requests for information from other states and from 
researchers in school effectiveness; and making presentations at 
national meetings. 

During the twielve months ending June 198A, TAs spent 202 days on SITIP. 
For each model, between A5 and 55 person days were spent on technical 
assistance, with the least amount being devoted to STL (which received 
additional help from the Johns Hopkins developers), and the largest amount to 
AT and ML (which had many more participating schools than STL or TV) . Indivi- 
dual TAs invested between 17 and 35 days each, with the veteran in each dyad 
spending slightly more time and taking lead responsibility. ^ 
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Impact 

Impact Is discussed below In terms of the team and each dyad. Relative 
success was Influenced by the challengers and perceived rewards discussed 
earlier. That Is. when a TA found SITIP participation personally and 
professionally rewarding, and also found ways to jre solve the challenges 
(particularly those relating to conflict of MSDE roles or conflict of 
state/local expectations), success In assisting local projects was high. From 
a local perspective. TA was valued when friendly colleglal relationships w.ere 
maintained and when the TA provided in-depth expertise to enhance local 
efforts or resolve problems. In terms of the particular purpose of the TAs 
for the I983-8A year (i.e.. helping LEAs take greater responsibility for their 
own projects), accomplishments varied, with the greatest single barrier being 
local turbulence (e.g.. staff reasslgnments. project expansion or redesign) 
which usually resulted In Increased demand for TA Involvement. 

In general. TA accompllphments Included: providing leadership for a 
statewide project; providing opportunities for local educators to share and . 
publicize their successes; applying strategies to facilitate implementation in 
new sites or expansion in old ones; maintaining networks among projects; and 
developing expertise themselves to apply not only to SITIP but also to other 



areas. 



In addition, Impact for each model Included: 

• AT ~ combining expertise in AT (process) and in mathematics 

(content), so that materials and training provided by MSDE reinforced 
instruction and curriculum at a time when state functional mathematics 
test scores indicated need for Improvement. Mean ratings across role 
groups of AT TA were 3.05. ranging from 2.94 (teachers) to 4.00 
(central office staff) Influenced by TA contact with a given role 
group.* 



* Ratings were on a five point scale with 5.0Q as most positive, 
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• ML helping local educators to help themselves. In one case 

simplifying an overly complex project and resolving old Issues » In 
othjsrs pressing for real .fidelity of Ifflplementatlon or providing more 
Indepth information; for all projects providing a catalogue of locally 
developea turrlcitlum and training materials. Mean ratings across role 
groups of ML TA were 3.52», with the greatest difference between 
school administrators (4.14) and teachers (3.29). 



e STL — helping Individual teachers by acknowledging their ef forts » 
providing materials, or providing opportunities for them to visit 
other classes and exchange ideas; encouraging real fidelity of 
Implementation by having project schools host visits from staff from 
other LEAa. The mean rating across role groups of STL TA were 3.33 
(ranging from 2.91 assigned by teachers to 4.75 by central office 
staff). Specific reference was made by some respondents to the lead 
TA for her friendly helpfulness. 

e TV encouraging Involvement of central office staff In local 

projects, (e.g., for the May 1984 Instructional Leadership Conference) 
and helping to improve organizational climate (e.g., through follow-up 
activities). The mean rating of TV TA across role groups was 3.20 
ranging from 3.02 (teachers) to 3.67 (school admlnistratorn) . 

The above examples relate to accomplishments resulting from TA -actions. 
Lack of action sometimes had a negative Impact, and was related to the role 
conflict of differing expectations.* For AT, some LE^s would have liked more 
on-site workshops. However, workshops were not cor.ducted when an LEA appeared 
to be taking Insufficient responsibility for a project. Also, one TA left 
MSDE in February, leaving the other with a heavy workload and little time for 
additional training. 

For ML, large LEAs expanding their projects would have liked more MSDE 
support (funds or person time) to help got more done, but did not get It 
partly because the policy was to fund each LEA equally (regardless of LEA 
size. Implementation strategy, or coiiq)lexlty of the model), and partly because 
the TA with appropriate expertise was "spread thin." 



* i rob] ems specific to lack of TA action for STL were not identified during 
the school year. 
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For TV, some LEAs would have liked follow-up workshops that provided more 
substantive training, and others stated (rather vaguely) that they wanted 
"better" TA. These LEAs did not get what they wanted, partly because the TV 
TAs believed that local staff should take greater responsibility for their 
projects. 

For all models, there were areas of need that c^uld have been addressed 
by the TAs, but they were not recognized or were recognized at the end of the 
school year (e.g., specific curriculum or organizational needs in AT, teaching 
strategies for "correctives" in ML, curriculum exchange and increased systemic 
involvement for STL, and a much greater emphasis on strategies to improve 
instruction — instead of the over emphasis on coding — for TV), 

Overall, energy and enthusiasm waned among TAs during the 1983-84 school 
year. However, some gains were consolidated, and there were activities or 
areas of high success for each model that would not have occurred without the 
TAs. Some lessons have been learned that are informing new projects. The 
recommended tasks that TAs set for themselves in June 1983 were addressed, and 
achieved to varying levels of satisfaction. 

Evolution of Technical Assistance 

This section reviews technical assistance from the spring of 1981 through 
June 1984. Attention is given to staff assignments, tasks and time alloca- 
tions, and challenges and successes. 

It is important to note tha' between December 1980 and June 1981, MSDE 
invested considerable effort in planning and training, conducting four major 
awareness conferences, planning summer training institutes, and assisting 
local teams in developing funding proposals and implementation plans. Evalua- 
tion of those activities indicated that dystematic technical assistance would 
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be needed If SITIP was to succeed. Therefore, the ICC made a connnltment to 
support TA beginning July 1, 1981 and assigned staff from their various 
divisions. Prior to that time, some people who subsequently became TAs had 
attended awarenes^ conferences or planning sessions. However, while a few 
anticipated some Involvement In general follov-up activities » none were 
lormally assigned to a given model until late June. Therefore, the Initial 
knowledge building activities of the TAs were not focused. 

Staff assignments . Table 13 summarizes the participation of the TAs from 
January 1981 through June 1984 » Indicating for each model the number of TAs 
and their usual role (e.g., for Active Teaching, ATI was a branch chief in 
basic skills, and both AT2 and AT3 were mathematics specialists In the same 
division). The extent of Involvement is Indicated over time and for critical 
events. 

The greatest staff stability was for TV, with the same two people 
involved for almost the full three and a half years. Each of the other models 
had at least one TA participating for at least two and a half years. There 
was most change in ML, with two "unofficial" delegates Involved for a short 
time, and "lead" responsibility switdhlng from MLI to ML2 In June 1982. 

When staff were assigned to a model they had to learn about the mode , 
about local implementation activities, about the norms of the TA system and of 
LEAs, and then had to learn how best they could provide assistance. All TAs 
found such learning difficult, with most difficulty experienced in the first 
few months of Involvement, especially by people who did not attend training 
events conducted by the model developers (activities *1 and *3 on Table 13). 
Some TAs (especially for STL and TV) initially thought they did not need to 
develop expertise in the model, but could focus more on prot^.esses of imple- 
mentation. While this caused no problems for STL (since developers readily 
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TabU 13 



Participation of Tadmical Assiitanta 

1981-1984 



Mod>l 


Incwtbenta' Ragular Role 


Extent of Involvement 


Active 
Teaching 

Maatttry 

Learning 

Student 
lean 
Learning 

Teaching 
Variables 




rjan" — 3iny Jan July Jan Juiy Jan Juiy 
1981 1981 1982 1982 1983 1983 198A 198A 


la Branch chief* basic akills, Inst. 

0 Specialiat. aatheaatics. Inat. 

3. Specialiat» aatheaatica, Inat. 

ia Branch chief* ataff dev.* CAA 
a. Specialiat. ataff dev, CAA 

ba Specialist » staff dev., UK 

0 Specialist » Title I» CUSP 
3a Specialist* inst. design, ITV 
0 Adainistrator, Project Basic 

2. Specielistt staff, dev.. Lib. 

3. Specialist, Mdia, Lib. 

0 AdainistratoT, Project Basic 
2. Specialist, staff dev.. ITV/C&A 


a i 

*1 *2 *3 *4 

.*1 *2 *3 *4 *3 

*3 

^ 

*1 *3 *4 *3 

% *3 

• i 

*3 

a 

*1 *2 *3 *4 *3 
» 1 

♦3 *4 • — 

*3 

*3 ♦A *3 

•3 



Kay: 

n4„< Critical Evanta Attandanca lolaa 

|sg!Sfficatlon and Accradit.tion Jl-a-aran... conf.r.nca. Ji'^^r.^fhr-Urd" ""aS^' 

CUSP.Co«p.n-.tory. Urban. Suppl.~nt.ry Progra.a I J , "a"'"* ~d" U id^UfilS iy 

I-TJlir-rrcri^^il Talaviaion ['l^:^^^^^^^ TA ^ ^ 

Llb-Llbr.ry S.rvica. a-fooMlly a.aignad a. TA 
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provided that expertise on request) » It had a negative Impact to some extent 
for all models, and effective TAs did Invest effort In learning when they 
realized the need. For most TAs, learning of all kinds related to the role 
was on«-golng and cyclical. Including theory and experience* Usually In the 
second year, everything ''came together'' and Incumbents "dlsco'^ered" how to 
Integrate technical knowledge (e.g., related to a model or curriculum subject) 
with training skills and organizational strategies. Learning was Individually 
determined, with some help provided by existing TAs to In-comlng staff (always 
perceived by newcomers as Inadequate) , and almost no formal provision made 
within the TA system (with the exception of small amounts of time during some 
moathly meetings when philosophy and strategies were explored or recent 
studies reviewed) c While many TAs considered learning as Inadequate, little 
was done to correct the situation, partly because regular roles made heavy 
demands on Incumbents' time, and partly because people often discovered their 
"need to know" after the fact. As "hindsight," 1984 TAs recommended greater 
attention to knowledge building, for themselves and for division directors who 
would thereby better understand the TA role and assign appropriate staff (with 
"field" experience and regular roles that could be readily Integrated with a 
given model) . 

Tasks* Table lA summarizes percentages of time Invested by TAs In the 
ten task areas, making comparisons across the three years from June 1981 
through June 1984 « The total number of person days Invested In Year 1 (June 
1981-June 1982) was 175, for Year 2 — 263, and for Year 3 202. With the 
exception of evaluation tasks, percentages of time Invested varied from year 
to year, and the specific nature of each task also varied. 
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Table 14 

TA Time Invested Over Three Years, by Task 




TMt t«lM 4aT« Talk Ar«at 

YMt VlOl 
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Variation in the number of days spent was influenced partly by local 
needs, but more by the incumbent TAs. Branch chiefs spent less time 
on SITIP, finding the tasks conflicting strongly with their adminis- 
trative responsibilities. Specialists in their first six to twelve 
months with SITIP spent less time, either because they were learning 
what to do or because they preferred working on the regular role 
assignments. 

Variations in the percent of time invested from year to year, and 
variations in the specific nature c»f a task area, were most strongly 
influenced by local needs. 

Administration . The nature of the task changed very little, 
consisting primarily of distributing and collecting forms from LEAs, 
allocating funds, and record keeping. The low investment (5%) in Year 
1 was due to the fact that much of this work had been done before July 
1981 by staff development branch staff and by administrators reporting 
to the ADS (before the TA system was organized). 

Planning . For this task, the low investment (3%) in Year 1 was also 
the result of a great deal of work done earlier by others. Planning 
related primarily to major training events such as the annual state- 
wide instructional leadership conferences and summer institutes. It 
also related to MSDE programs similar to SITIP, such as URATE (a 
training/ implementation project involving colleges and universities), 
and to LEA planning. The nature jf planning changed little from year 
to year. 

Knowledge building consisted of reading and discussing research and 
practice on the models, classroom and school effectiveness, and 
planned change, and developing strategies to apply that knowledge. In 
Year 1, TAs did not do this, relying on their existing knowledge and 
experience. (Some did attend the 1981 summer institutes, building 
somewhat on knowledge of the models acquired at the January 1981 
awareness conferences, but they did not believe at the time that they 
needed to become technical experts, thinking of themselves more as 
process facilitators.) In Year 2, several TAs (especially in AT and 
ML) decided they "needed to know" and invested 10% of their time 
building knowledge. By Year 3, with the exception of the two new TAs, 
this task evolved into keeping up to date or learning about specifics. 

Mater ials development / ident if icat ion related primarily to training. 
TAs made video tapes and/or distributed developers' or ASCD tapes to 
LEAs, developed handouts for workshops, and distributed copies of 
relevant articles. In the first year, TAs did virtually none of this, 
relying instead on materials distributed to LEAs by developers, tn 
Year 3, for ML and STL, TAs compiled and distributed catalogues of 
LEA-developed curriculum and training materials. 

Training . In Years 1 and 3, when summer institutes were conducted, 
22% of TA time was invested in training. This task included facili- 
tating sessions at training events, conducting workshops, working as 
partners with local staff in LEAs, and conducting follow-up sessions 
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which (for STL) Included classroom visits. In Year 2, most TA 
training was conducted for new SITIP projects or in LEAs where staff 
reasslgnments had resulted In loss of the local trainer or advocate. 

• General support Included logistical and affective help, often over the 
phone or In brief encounters (at SITIP and non-SITIP events). Logist- 
ical support consisted of providing information, clar-*.fylng tasks or 
Issues, making arrangements for site visits, or linking staff or 
project teams with common Interests. Affective support consisted of 
acknowledging and publicizing successes, and maintaining a positive 
program orientation. TAs recognized individual teacher's strengths, 
and encouraged networking by inviting one project to learn from 
another's success. More time was spent o\ general support in Year 1 
because local staff were getting accustomed to the project. 

e Visiting sites took almost twice as much time in Year 1 (AO^) as in 
Year 3 (24%), and the least time in Year 2 (1531), although in each 
year TAs visited each project at least twice. In each year, TAs 
observed classroom application (for all models except TV), and talked 
with representatives of all three role groups about the model and its 
use. They also engaged in trouble shooting and planning for project 
improvement, expansion, or termination. Sometimes they participated 
in local staff meetings or training sessions, or met with adminis- 
trators to resolve difficulties or share successes. Most help was 
needed at the beginning of a project. Also, in the first year, TAs 
were themselves learning how to integrate tasks and manage time most 
effectively. Those two factors contributed to the high Investment in 
Year 1. Year 2 was a "maintenance/consolidation" period for veteran 
projects and TAs learned ways of helping LEAs to help each other. 
Year 3 required decision-making, and TAs again spent more time 
"on-site." 

• Evaluation was undertaken primarily by RBS. TAs invested about the 
same amount of time each year distributing and collecting surveys and 
related materials, keeping their own records, and assisting LEAs or 
RBS in data collection or evaluation design. 

• Communication was defined as the interaction among TAs, other MSDE 
staff, or RBS about SITIP, and included the formal monthly TA meetings 
or reporting sessions to the ICC as well as more Inforraal communica- 
tion. This activity took as much as 13% (Year 2) to as little as 8% 
(Year 3), and was related to knowledge building and dissemination. 

• Diaae mlnation was defined as involving or informing o:hers about 
SITIP, b"eyond those intended in the original plan. The greatest 
investment (HZ in Year. 2) Included presentations at regional and 
national conferences, and training for non-SITIP iraplementers.* 



* An LEA implementing one model might ask for a workshop ot. another model, or 
a TA (in the "regular" role) might be invited to conduct a workshop and 
choose (as the major content area) a S i TIP model or proceso. 



While all TAs engaged In all task areas to some extents administration 
and planning tasks were most likely to be undertaken by the administrators 
usually reporting to the ADS. Other regular roles did not greatly Influence 
task allocation* For Instance^ contrary to what was expected, library media 
staff spent no more time on materials than^other TAs, nor did staff develop- 
ment branch staff spend more time on training than other TAs. Local program 
needs and the Individual TA*8 Judgement of how he/she would respond were the 
strongest Influences over the three years. 

As TAs looked back over the period, they Identified changes that should 
be made If a project like SITIP was Initiated again. Those changes are 

« 

presented here as recommendations* 

• Educate MSDE division directors to build SITIP commitment, and to 
assign staff most likely to be succe&^sful. 

• Assign TAs who can do double duty In field work and/or Integrate SITIP 
tasks with regular role responsibilities, recognizing that most TA 
time Is spent on site visits and training. Avoid TAs who lack commit- 
ment, resist the assignment, or are delegated tasks In which they have 
little knowledge or skill (especially expertise In classroom teaching 
and project management). 

• Develop TA knowledge and skill In organizational analysis; techniques 
and strategies of project planning, management, support, and evalua- 
tion; application of research on planned change; the model (innova'- 
tion) and its relationship to current state and local curriculum and 
instruction priorities. 

• Involve TAs in knowledge exchange, informal observation of each other, 
and exploration of organizational differences and assumptions (in MSDE 
and the LEAs) in order to understand and use their autonomy 
effectively. 

• Ensure that the 15% to 20% time per per .on per year to SITIP is a real 
commitment (by TAs and their supervisors), and that (in addition to 
intensive training and planning ahead of time) 40 to 60 person days 
are spent on support for each model each year. 

• Ensure that in working with LEAs, TAs assist local staff in clarifying 
their purposes; ensure central office staff involvement and support by 
building a sound knowledge base and visiting schools and classrooms 
together; spend time on-site early in the project to build a cotmnon 
understanding of the project and state/local roles and responsibili- 
ties; develop and maintain good working relationships with local 




staff; use site, visits to engage in cross-hierarchical 
problem solving and not pro forma monitoring; and apply 
positive pressure and sound knowledge to ensure rich 
fidelity of implementation. 

• Explore ways for TAs to build bridges across tasks within MSDE. 

• Invest more time and enthusiasm in linking implementation of 
models, and/or linking SITIP (knowledge, beliefs, projects, 
people) with similar activities or task areas. 

These "hindsight" recommendations were made on the understanding that 

they would be in addition to current TA practices that, overall, have had a 

positive impact on SITIP implementation. 

Sumtaary and Conclusions 

MSDE initiated a statewide instructional improvement program that offered 
LEAs choices of exemplary models. The state department established an organi- 
zational structure across divisions, using existing expertise and mechanisms 
to provide coordinated support and to facilitate communication. 

Planning activities invited participation across hierarchies and organi- 
zational units at the state and local levels. On-going communication about 
SITIF interacted with related activities in other program areas so that the 
"message" was clear, consistent, and widespread. The "message" was the 
research-based knowledge on classroom and school effectiveness, and planned 
change. 

Through various state-sponsored training activities, representatives of 
all educational communities in the state had the opportunity to learn about 
this knowledge base. LEA teams contributed to the training and learned from 
each other and from "experts." Training activities were very well received 
and were follovied up by assistance from MSDE staff. 

SITIP TAs worked aa a team to contribute to planning, design training, 
and provide on-site assistance related specifically to the models. They also 
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monitored plans, administered grants to local projects, and coordinated evalu- 
ation and dissemination activities so that data-based improvements could be 
made and successes could be widely spread. Their philosophy of building 
capacity in locally "owned" projects (rather than In monitoring compliance of 
a state "owned" project) was a positive influence on TA activities and 
outcomes. 

As SITIP continues for the 1984-85 school year, so TA will continue, 
although the number of Incumbents will be reduced to reflect reduced local 
needs. It appeared that by June 1984 technical assistance for SITIF was 
institutionalized as long as a real local need existed. However, as more LEAs 
developed their own training capacity aad learned to help each other, MSDE 
assistance became less crucial. The notion of general technical assistance 
becoming institutionalized in MSDE instructional initiatives appeared to be 
conditional. That is, if a program director understands the benefits of TA, 
and allocates funds to support the person time needed, the assistance role 
will continue. (For instance, the proven value of Project Basic facilitators 
suggests that the role will contime, although incumbents may focus on a 
different state priority.) However, as one SITIP TA pointed out, "It's a 
risky investment if you don't understand it," and most members of the ICC 
appear moce likely to continue to assign staff and develop programs in more 
traditional ways. Yet, individuals who provided technical assistance for 
SITIP will incorporate some of that knowledge and skill in their regular Jobs. 
Also, in some other programs, aspects of technical assistance and parts of the 
SITIP design became apparent (e.g., URATE is a prograta like SITIP involving 
colleges of education). 
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SITIP was Influenced (in 1980 as it was designed) by successful elements 
of the Professional Development Academies and Project Basic. By 1984, SITIP 
had. in turn. Influenced both of those programs. Organizational learning 
occurred, to the benefit of LEAs. IHEs. and MSDE. Instructional improveuient 
in over 180 schools was facilitated by MSDE initiatives. 
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V. LOCAL IMPLEMENTATION AND IMPACT 

This chapter describes local Implementation and Impact of the SITIP 
raodf^ls for the 1983-84 school year. The overall questions addressed vere: 

• What was the nature and extent of local implementation? 

• To what extent were SITIP activities institutionalized? 
Additional questions addressed included: 

• What were the scope, intensity, and fidelity of implementation? 

• What were the roles and responsibilities of local participants? 

• What was the impact on schools and school systems, on educators, on 
students, and on insteruction in general? 

• What decisions and cl^ges were made? 

Finally, the answers to these questions were synthesized to determine: 

• How did the various factors interact to influence project success? 

In the following pages, some general background information is provided, 
the local Implementation and impact of each model is discussed, and some 
conclusions and Implications across models are presented and discussed.* 

General Background* * 
This se(^tion summarizes the conditions of local implementation for all 
the models, first describing state and local goals, then reviewing 
expectations, planning and training, implementation strategies, and the scope 

0 

jOf the Implementations. 

/ 

\ State and Local Goals 

/ From the state perspective, successful Implementation of SITIP would be 



* Case studies of each LEA are presented In a separate report. 

Please refer to the Summary of Findings: December 1980 to June 1983 
presented In Chapter II of this report for additional background 
Information. 
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use of one or more of the models by many teachers in many schools in all LEAs. 
The implementation would improve instruction (thereby improving students* 
achievement and attitude toward learning), increase teachers' effectiveness, 
prove to be useful for both elementary and secondary instruction in various 
academic subjects, increase administrators' ability to manage planned change, 
and be carried out in such a way that productive working relationships were 
maintained across role groups. Finally, as state funds were gradually 
withdrawn, MSDE hoped that local SITIP projects would be Institutionalized, or 
terminated if instruction had net been improved, (with that decision based on 
project results discussed by all role groups). 

The goals of improved knowledge, skills, and attitudes for students and 
teachers were expected by most LFAs. Organizational harmony and administra- 
tive skills were not overt local goals. Also, very filw LEAs were initially 
interested in promoting widespread use or systematic institut'ionalization, 
although some districts did address those goals after the first year. 
Expectations 

Since SITIP involvement was voluntary. It was important for LEAs to 
understand what was expected of them if they chose to participate. 
Each LEA was expected o implement a model with "fidelity," to Involve 
cross-hierarchical teams^lji-j^lanning and implertentation, to send representa- 
tives to state-sponsored training events, to Interact constructively with TAs 
and other LEAs implementing a given model, *and to provide information relevant 
to program evaluation and student assessment. Nine LEAs also agreed to serve 
as pilot sites for additional data collection by RBS. Each LEA received up to 
$5,000 In state funds for Year 1, and up 'to $3,000 in each subsequent year on 
condition that the local system provide matching grants. (For Year 4, state, 
funds were available only to expanding projects,) ^ 
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LEAs were free to choose the model (s) most likely to meet local needs, 
and to specify their own Implementation strategies and the outcomes they 
exacted. Also, eaclj LEA could change plans (i.g., reduce or expand the scope 
of Implementation, terminate a project, or adopt another model), and, in 
making such changes, were encouraged to make data-based decisions (e.g., 
referring to students* test scores, teachers' reports, or RBS' studies). If^ 
the LEAs reduceid their workscope or procrastinated, they were offered 
assistance to meet their own goals or given the choice of returning state 
funds for that year. 
Planning and Training 

MSDE sponsored many planning and training activities, expected local 
Involvement, and encouraged l^ocal educators to conduct similar activities.* 
Local participation in state sponsored activities for the 1983-84 year was 
fairly good, although no follow-up workshop was conducted for AT, and 
attendance for STL workshops was poor for those LEAs not using SITIP funds for 
that model.** The Involvement of a cross-hierarchical team (a central office 
person, one or two principals and two or three teachers from each of the 
principals' schools) was important in key planning and training events, since 
it helped create a shared understanding and Increased local ownership and 
commitment. In projects where team staff were reassigned, commitment vras 
reduced, there was confusion about project status, and (sometimes) the purpose 
and value of the effort seemed to be lost. These setbacks could be overcome 
if planning and training activities were carried out, and TAs often assisted 
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See the section on Training In Chapter IV. State Initiatives and 
Assistance^ for a daiscrlption of state sponsored events. 

Three LEAs that originally adopted STL plus another model investted In the 
second models leaving educators to Implement STL Independently . 
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LEAs in this area. Where projects progressed more easily, planning and 
training were conducted locally, sometimes with TA assistance, and — for best 
results — Involved administrators as well as teachers. 
Implementation Strategies 

During Year 1, it became apparent that staff interest was the most 
influential factor in selection of the model and design of the implementation 
strategy. (High teacher interest was linked to Individual classroom needs, 
but high central office staff interest was linked to district priorities.) 
While work at the school site was strongly influenced by the complexity of the 
model, work across the LEA (how much, how it was shared, how workloads shifted 
among role groups over time) was determined by the strategy. In other words, 
strategies requiring more work across role groups (and leading to widespread 
implementation) were Initially selected in LEAs where administrators believed 
that SITIP could address a local priority. In some LEAs more than one model 
was adopted, sometimes with different strategies for each, and some models 
were adde^ or deleted after the first year. When implementation was 
successful, a switch was sometimes made to a more work- intensive strategy. 
When implementation was less successful, a switch was sometimes made to a less 
work-intensive strategy or the project faded away. 

The four strategies designed or selected by LEAs are summarized below. 
Table 15 relates strateg'^es to scope of irapleraentation for each model over the 
three years. 

• District-wide . All schools at a given level (usually elementary) 
were Involved, with the selected model used for a given subject 
all the time by participa*"lng teachers (at least three per school 
in the first year, all tei^'^hers in subsequent years). This 
strategy required the most work from the most people, with 
central office staff enthusla^di and effectiveness important for 
success. Two projects began with this scrategy, and by June 1984 
a third was also Implementing SITIi^ dif^trlct wide. The largest 
project Involved 33 schools • 
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Table 15 



Sqope and Strategies of Implementation: 
All Models Over Three Years 
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Strategies: DW - District-Wide 
PD « Pilot/District 
CB - Capacity Building 
LS » Lighthouse School 
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• Pllot/Dlstrlct . One to three schools were Involved the first 
year, with strong central office support for school-based 
activities. Evidence of success led to greater administrative 
involvement and, in some cases, use of key teachers as turnkey 
trainers. This strategy was the most feasible, especially for 
complex models. Five projects began with this strategy, and 
eight were using it by June 1984. The largest number of schools 
involved in a pilot/district LEA was 28. 

• Capacity Building . Training was conducted by the LEA team that 
participated in MSDE institutes. Teachers volunteered to "try" 
the model. There was no formal commitment to follow-up by 
administrators, although where this strategy was effective an 
administrator did "energize" the project. Five projects began 
with this strategy, of which three faded out during the second or 
third year. By June 1984, there were three capacity building 
projects (one having switched from a lighthouse strategy) with 15 
schools involved in the largest project. 

• Lighthouse . A single school was involved and no commitment was 
made by central office staff to advocate further use or initiate 
planning or training for other schools. Success was usually 
broadcast informally. This strategy put the greatest burden on 
school staff. There were 20 lighthouse sites Initially; lA by 
the end of Year 3, seven having evolved into pilot/district sites 
and one into capacity building. Two ending as lighthouse sites 
had begun with other strategies. By June 1984, the largest 
number of schools involved in a lighthouse LEA was three. 

Over the three years of implementation, there were 31 projects in the 2A 
LEAs; 19 maintained their initial strategies; three maintained their strat- 
egies for a year or two and then faded out; 10 changed strategies, with eight 
increasing their work intensity and scope of implementation, and two reducing 
their intensity and scope of implementation. For widespread implementation, 
the lighthouse strategy was least effective, but vas successful (from a 
small-scale perspective) when the model matched a principal's priority. 
Capacity building was least effective for maintaining systematic 
implementation, but did increase teachers' knowledge of an alternative 
instructional mode"... Overall, the pilot/district strategy was most effective, 
particularly for complex models in large LEAs. The district-wide model was 
successful with less complex models if attention was paid to building the 
commltmenL of school based staff. 



75 



85 



Scope 

Scope of Implementation was related to the strategy used, initially 
reflecting central office staff's Interest, but in some cases, being later 
influenced by project outcocss. The number of schools involved in Year 1 was 
64, in Year 2 there were IM, and by the end of Year 3 there were 185. About 
66% of the schools were elementary. Over the years, some LEAs with 
more than one project dropped or added models (in terms of funding or ctmtral 
office support). Also, the five LEAs not initially involved did adopt: models 
at the beginning of Year 2 (and benefited from the "veterans'" experience). 
All grade levels and academic subjects as well as vocational-technical classes 
were involved. For AT, most participating teachers used the model all of the 
time for mathematics. ML was also used quite Intensively. However, STL was 
used sporadically, and TV only directly influenced teachers' behavior (in the 
"content" variable) for language arts and mathematics at two schools. The 
scope, intensity, and fidelity of the implementation had a direct influence on 
a model's impact on students. 

It is apparent that while all local educators hoped that teachers' 
knowledge and skill in instruction would Improve for the benefit of the 
students, the expectations, processes, and levels of energy invested varied 
considerably, with project outcomes directly reflecting those Influences. 

Active Teaching (AT) 

This section describes the local implementation of AT. The inlormation 
presented here is based on various data sources, including survey 
questionnaires completed by 126 local educators (11 central office staff, 24 
school-based administrators, and 91 teachers). All LEAs are represented in 
that population. Approximately 64% of the responding teachers used AT in 
elementary schools. Discussion focuses on: planning; the scope, intensity, 
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and fidelity of use; the roles and responsibilities of implementers; the 
outcomes in terms of impact on students, teachers, schools, and school 
systems; and influences and plans for Year 4. The section summary reviews 
major changes over the three years if implementation and presents some 
recommendations for the future. 

Planning 

As stated earlier, AT is a simple model of direct instruction, often 
I perceived by experienced teachers as a traditional lesson structure. It was 

I designed to be used for mathematics, but has also been used in other subject 

areas. Due to the developer's subject area emphasis and the fact that two of 
I the three TAs assigned to AT over the three years were mathematics 

specialists, many LL.s focusfcd on .mathematics improvement. In some cases, 
I plans expanded in Year 2 because scores for tht statewide competency examina- 

I tlons (graduation prerequisites) were published, and central office staff made 

mathematics improvement a priority. 
I An analysis of local plans for the 19b3-84 school year identified LEA 

objectives at the beginning of the year (September 1983), and the extent to 

which project coordinators considered objectives to be achieved at that time 
I and also at the end of the year (June 1984). (See Table 16.) In each case, 

' the percent of LEAs that "hoped for," "partly achieved," or "achieved" each 

I objective is indicated. There were nine objectives identlfi«|d, many partly 

achieved at the beginning of the year (having been addressed in Years 1 and 
I 2), and some maintaining that status in June (especially in LEAs expanding 

projects to additional classes or schools). While the first five objectives 
* might well be expected for AT, curriculum alignment (#6), and tlme-on-task 

I (#8), are not part of the model but were strongly addressed In MSDE training 

activities over the three years and were adopted by LEAs striving to improve 
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Status of Local Objectives, 1983-8A: Active Teaching 
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Note. Total number of LEAs implementi;-j Active 

Teaching equal?* 9. N equals the number of 
LEAs addressing a given objective. 



No data are available on the status of objectives in September 1983 for Garrett and Warihington 
counties. Somerset County began implementation after September. No data are available for 
'••a'" 1984 for Washington County. 
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quality of Implementation • Overall objectives addressed focused on classrootD 
Improvement, and indicated a shift from earlier attention to orientation and 
training. 

Scope » Inten s ity, and Fidelity of Use 

Scope of implementation indicates the numbers of districts, schools, 
teachers, and so forth i^hich w«re involved* Intensity relates to the amount 
of time a given teacher or class used AT. Fidelity it; the extent to which 
teacherr> implemented the components of the model as Intended by the developer. 
Scope, intensity, and fidelity in the classroom were influenced by 
administrators' investment of time, and the kinds of activities they carried 
out. Influenced by the implementation strategy employed, these dimensions 
indicate the nature and extent of use. Table 17 summariziis the use of AT for 
Year 3. 

Scope . Nine LEAs implemented AT: two useo the district wide strategy; 
two were pHot/dlstrlcts (one of which evolved from a lighthouse school); one 
used a capacity building approach; and four were lighthouse sites. Somerset 
and Washington were "new" sites, the former using AT as an improvement 
strategy for TVp the latter stimulated by involvement in a unlversltv study 
comparing AT with STL. While a few schools were added during the year, the 
most notict^able expansion was the increase In the numbers of teachers Involved 
— almost doubling from 872 to 1710, More than 9,000 additional students were 
involved, bringing the total to over 41, POO, most of whom were In elementary 
mathematics classes. This expansion within schools was facilitated by key 
teachers (i.e., one or two teachers from eanh srthuol or departir'^nt vlio were 
trained a^nd coached in Year 2), who t( en provided coaching for their 
colleague':, trained by administrators in Year 3. Across the state, AT wps the 
moat widely implemented model, used in 51% of the SITIP schools hy about 63% 
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Scope of Implementation, September 1983 and June 1984? Active Teaching 



LEA 


Strategy 


Dimensions 




Caroline 


LS/PD 


// of Schools 


Ty 


pe 


# of Teachers 


// of Students 


Subject Areas 


Sept 83 ' 


June 84 


Sept 83 


iune 84 
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* Includes some duplicates. 

+ Began implementation after September 1983. 

St rategy ; LS»Lighthouse school Type ; E=Elementary school 
PD=^Pilot/district j/M=Junior high/middle 

CB=Capacity building H=High school 

DW==District~wide 0=Other 



Sublect Areas : R/LA«Reading/ language arts 

M«Mathematics 
Sc^Science 
SS^Social studies 
0==Other 
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of the SITIP teachers. Twenty-eight of the 93 schools were secondary, 65 
elementary. 

Intensity . While five LEAs began In Year 1. two in Year 2. and two more 
In Year 3» and by June 198A, somfc teachers hafl therefore used AT for three 
years, the average amount of time for teachers' Involvement was 1.55 school 
years. This Is not surprising given that the number of teachers almost 
doubled each year. During the 1983-8A school year, teachers used AT for 8.7 
months, with the lowest LEA mean 6.37 months (in a county experiencing high 
environmental turbulence due to staff reasslgnments) . AT was used for all 
mathematics Instructiort by participating elementary teachers, for all 
mathematics instruction for at least one grade level in secondary classes, and 
for some classes in the three LEAs where the model was used for other subject 
areas. 

Fidelity . AT as designed requires Implementation of six components; (1) 
pre-lesson development (homework review); (2) lesson development" (presentation 
of new information); (3) guided practice; (A) independent practice; (5) 
homework; and (6) weekly or unit review. Of the 90 teachers responding to the 
general survey, 91% carried out all six components regularly. All teachers in 
four LEAs carried out all components. In three other LEAs, all teachers 
carried out five components, with a few not implementing weekly or unit 
reviews. The two counties with lower fidelity levels (both lighthouse sites) 
had less administrative support than did other LEAs, one because of environ- 
mental turbulence, and the other because the project was in its first year and 
roles and responsibilities were being clarified. 

Administrative investment . The amount of time administrators invested in 
SITIP during Year 3 was 16. A5 days, with central office staff spending more 
time (mean - 25.5 days) than school-based administrators (10.17 days). 
Activity areas, in order of priority allocation of time, included: (1) 
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Inservlce, (2) monitoring/evaluation, (3) general support, (A) communica- 
tion/administration, and (5) ^Issemlnatlon/expanslon.* Where central office 
staff were Involved in training, inservice was a high investment, but when 
that task was undertaken by teachers, administrative priority shifted to 
monitoring. In the two LEAs where implementation began in Year 3, and teacher 
training (in the single lighthouse schools) was conducted at the MSDE 
institute, administrators invested in monitoring instruction. 

In general, the quality of implementation was high, particularly for 
elementary mathematics. The amount of time spent on AT was also high, 
providing a good opportunity for implementation to make a real difference In 
classroom instruction. 

Roles and Responsibilities 

The SITIP design encouraged the involvement of cross-hierarchical teams. 
By the end of Year 2, it was apparent that: (1) teachers involved in MSDE 
training activities often became instructional leaders or "key" teachers who 
coached others in their schools; (2) school based administrators of lighthouse 
sites and others trained by MSDE usually were supportive of their teacheri and 
facilitated SITIP implementation; and (3) central office staff trained by MSDE 
often trained others in their LEAs and designed and maintained implementation 
and expansion activities. Locally trained educators could become advocates 
and Invest time and energy if certain procedures were used to build commitment 
and provide logistical support. However, project success depended on very 
careful coordination of various activities and "messages," and could be 



* See Roles and Responsibilities for a more detailed discussion of activltins. 
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seriously threatened by environmental turbulence. 

In Year 3, project staff needed to correct errors, ^consolidate successes, 
and make data-baaed decisions about use of the model as state funds were with- 
drawn. Local staff were encouraged to apply the recommendations made in 
earlier SITIP reports, with particular attention to central office staff 
InvolveiPent. The discussion below describes role group activities more 
specifically in terms of Interactive support and' leadership be'havlors. 

Interactive support s Support among LEA participants Included exchanging 
information and materials, providing formal training, coaching and trouble- 
shooting, managing logistical arrangements, and recognizing successes. 
Support from MSDE consisted primarily of training and technical assistance. 
Survey respondents rat^d the support received (on a five point scale from 
1.00-very poor to S.OO^exceilent) , and the mean ratings assigned by each role 
group are presented in Table 18. Overall ratings ranged from a low of 2.49 
(for developers) to a high of 3.76 (for school-based administrators). The 
developers of AT had no interaction with LEAs in Year 3, and received low 
ratings from all role groups. MSDE staff Interacted primarily with central 
office staff, from whom they received their highest ratings. Among LEA role 
groups, teachers received good ratings from all, and (as in previous years) 
assigned lover ratings than did bther role groups. 

Ratings within each county Indicated the following. 

Si 

• With the exception of one LEA (Garrett), teachers were awarded above 
average ratings, reflecting their steady hard work in implementation. 

• With the exception of one LEA (Garrett), school administrators w6re 
awarded above average ratings, with scores higher than their teachers 
in four LEAs where principals were particularly active (Caroline, 
Cecil, Somerset, and Wicomico). 



* "Messages'* of implementation Included overt statnments of enthusiasm for the 
project and acknowledgement of teachers* successes and principals' invest- 
ments, as well as more subtle notions such as sincerity in considering 
points of view and building shared understanding and trust about: project 
purpose. 

ERIC ' 8394 




f 

0^ 



Table 18 « . 

Perceptions of Support Received: Active Teaching, 1983-84 





SuppoxU: Groups 




Teachers 


School 
Administrators 


Central 
Office Staff ; 


MSDE 


Develop rs 


Respondents 


M 


Meanft 


M 


Means 


M 


Means ' 


M 


Means 


M 


Means 


Central Office Staff 


9 


4.00 


9 


3.67 


9 


4.00 


10 


3.80 


9 




2.67 


School Administrators 


22 


4.04 


18 


3.89 


22 


3.41 


22 


3.27 


18 




2.22 


Teachers 


88' 


3.53 


89 


3.74 


84 


2.94 


81 


2.90 






2.53 


Total 


119 


3.66 


116 


3.76 


115 


3.11 


113 


3.05 


98 




2.49 



Mean ratings range from a low of 1.00 (poor) to a high of 5.00 (excellent). 



95 



9i) 



. only in four LEAs (with district wide or pilot/dis rict stra egies) 
d?d central office staff receive above average ratings (Caroline 
?ecil! Harford, and Wicomico) in all of which training was conducted 
and supervision carried out using AT guidelines. 

. Low ratings reflected low interaction. When that was apparent for all 
iocal role groups, it signaled project decline. When apparent only 
for central offic; staff! it reflected a "lighthouse" implemen atxon 
strategy where project maintenance was the school's responsibility. 

In comparison to previous years, mean ratings across LEAs were somewhat 
lower for all role groups. However, this might be expected since most projects 
were integrating AT into routine activities for first and second year teachers, 
and also expanding to additional schools or teachers, thus "spreading thin." 
in general, in third year projects, energy levels were lower, which reduced 
interactive support among local participants. 

AH..ni.trative leadership. Behaviors found to be important influences* on 
project implementation and institutionalization were specifically defined, and 
participants were asked to assign ratings on each behavior for central office 
staff au school-based administrators. (See Table 19.) 

Affective leadership behaviors were strongly evident, with both school- 
based administrators and central office staff demonstrating commitment, 
advocating the value of AT. and providing support by demonstrating Interest and 
enthusiasm and recognizing teachers' success. Mean ratings were lower for 
central of.lce staff because that role group had less responsibility and 
visibility in the five lighthouse sites than did their peers in LE.s with 
broader implementation strategies. 

Logistical leadership behaviors were evident to some extent for both role 
groups. Higher ratings assigned to school administrators (all above 3.02) 
were influenced by proximity and visibility to teachers and by the fact that 
principals did carry out leadership tasks at all sites, while central office 
staff were not as heavily involved (especially for the lighthouse sites). 



Table 19 



Administrative Leadership Behaviors: 
Active Teaching, 1983-84 



1 

oenav I o 


Mean Ratings Assigned 


To Central 
Office Staff 
N « 9A 


To School Based 
Administrators 
N » 100 


t 

At rectlve 






Demonstrate commitment 


4.06 


4.55 


Provide support 


3.45 


4. 12 


LoRlstlcal 






Press for fidelity 


2.77 


3.02 


Press for Intensity 


2.95 


3.28 


Provide assistance 


3.39 


3.80 


Coordinate LEA communication 


2.97 


3.19 


Coordinate school communication 


2.81 


3.57 


Implement data-based 


2.76 


3.09 


decl&lon-maklng 



Scale ranges from 1.00 (not at all) to 5.00 (no a very large extent). 
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Given the fairly rout.nlzed status of AT at most sites, a "moderate" press for 
fidelity and Intensity (a rating of 3.00) is acceptable. Low ratings on those 
two behaviors (at some specific LEAs) were awarded when other evidence 
Indicated project decline. The relatively high ratings for assistance 
indicated provision of information, materials and training, and effective 
responses to requests for help. Since this administrative behavior (assist- 
ance) was strongly encouraged by MSDE (and It influenced teachers' success), 
these ratings are positive. School communication ratings were good, but mean 
central office scores were (again) influenced by lower scores for lighthouse 
sites. Data-based decision-making for SITIP was about the same as for any 
other local project: MSDE had hoped that this behavior would increase. 

In two LEAs, leadership was Initially undertaken by teachers. The 
reassignment of a teacher leader upset the project, and further staff changes 
(accompanied by a somewhat low commitment) led to project decline. In the 
second LEA, the teacher leader maintained effectiveness in his school, but 
MSDE and central office involvement was necessary for expansion to other 
schools to occur successfully. For AT, and for other models, teacher leader- 
ship was effective within the home school with principal support, and in other 
schools only with considerable support and assistance from central office. It 
was extremely difficult for a single teacher to provide leadership (unless 
released from teaching responsibilities); it was easier And more effective 
when two or three teachers worked as instructional leaders. 

For AT overall, interactive support and leadership was good, and appro- 
priate to the implementation strategies used. Effective expansion was 
possible only when central office staff were involved. Good classroom imple- 
n intatlon was reinforced when supervisors looked for AT during their classroom 
observations. 
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Outcomes 

AT had an Impact on students, teachers, schools, and school systems. 
While some LEAs voluntarily provided summaries of assessments of students' 
attitudes and academic achievement, in most cases, educators reported their 
perceptions. For Year 3 of the study, greatest attention was given to 
outcomes relating to institutionalization. 

Students . All three LEA role groups believed that students benefited 
from AT. (See Table 20.) They all agreed th^t time-on-task Improved and 
students enjoyed AT and learned more. Students understood the AT structure, 
and 40% of survey respondents found that learning skills improved, 33% 
reported improved grades or test scores, and 26% said that students' 
confidence and enjoyment of mathematics improved. Specifically, educators 
volunteered the following: 

"Students. . .master material quicker. .. retain more... are 
accountable for their learning. . .have greater confidence... 
have no time left to create discipline problems." 

In one county, over 1,200 students responded to an attitude survey, with 
mean ratings above 3.89 for all items (on a five point scale ranging from 
1.00-not at all, to 5.00-ye8, a lot), indicating they understood and enjoyed 
the AT lessons, learned well, and got better grades than in lessons where AT 
was not used. Overall results agreed with those of Year 2, although ratings 
were slightly lower for Year 3. 

Another county, having used AT (and a new curriculum) in mathematics for 
three years, reported results for grades 3, 5, and 8 en the California 
Achievement Test. Scores increased each year for each grade, with percentile 
ranks of 68, 71, and 69 (for grades 3, 5, and 8, respectively) in October 1983 
when elementary students had had two years of AT instruction, and middle 
school students had had one year. These scores compare well with October 1981 
scores of 62, 63, and 61 percentiles for the respective grades, before AT. 
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Table 20 



Instructional Impact as Perceived by 
Survey Respondents: Active Teaching, 1983-1 98A 




Instructional Value 



Works in classroom. 
Is worth the work It takes. 

Impact on Teachers 

Teachers enjoy It. 
Teachers have Increased knowledge. 
Teachers have increased skills. 

Impact on Students 

Students enjoy it. 

Students" achievement has increased. 
Students are learning and retaining more. 
Students' general behavior is better. 
Students* are taking more responsibility 
for their own learning. 



3.90 
A. 30 
A. 30 



A. 00 
3.70 
A. 00 
3.70 

3.60 



A.OB 
A. 00 
A. 00 
3.71 

3.92 



3.97 
3.86 
3.8A 
3.70 

3.A6 



4.06 
A. 01 
A. 05 



3.99 
3.87 
3.89 
3.70 

3.56 



igly disagree) to 5.00 (strongly agree) 



Mean ratlnKS range from 1.00 (stroaL . „ . 

Mean racing b ^.^^^^^ Administrators; T - Teachers 



CO 



Central Office; SA 



vol 



Teachers . -Peachers' knowledge of effective teaching and skill in 
instruction improved. Experienced teachers found AT "easy to use" and appre- 
ciated the "systematic perspective." New teachers found that AT provided a 
"structure for organizing lessons," and increased their confidence. Teachers 
said that they "made better instructional -decisions" and "improved time-on- 
task." Many appeared to agree with the one who said, "This is the way that 
math should be taught." 

In elementary mathematics classes observed, teachers were extremely well- 
organized. Objectives were clearly specified, and curriculum,*" instruction, 
and on-going assessment were well-aligned. Time-on-task was high, even when 
students used manipulables, since expectations were obviously established and 
management of materials went smoothly. Teachers maintained student attention 
without gimmickry, and provided various kinds of learning opportunities, using 
the chalkboard, overhead projector, number cards for total class individual 
response to guided practice, worksheets, and assorted games for independent 
practice. Succeaa rate was good, partly because teacher aides coached in- 
class, partly because ability grouping was used to some extent, and partly 
because teachers provided systematic review and practice, attending to prior 
* learning and varying learning styles. 

The timeliness of AT, given state-wide competency prerequisites for 
graduation and the national concern over mathematics instruction, plus the 
expertise of MSDE TAs and LEA supervisors, increased the press on teachers to 
do well. Classrooms serving as pilot sites were busy and constructive. 
Teachers' adherence to AT, their use of a variety of instructional materials, 
and their attention to student mastery was impressive. While not all such 
activity can be attributed to AT, some can, and acknowledgement given to 
teachers' efforts also increased their enthusiasm. 
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Impact on students and teachers combined was defined as Instructional 
gain. An analysis of variance showed significant differences between the 
counties on instructional gain. (See Table 21.) St. Mary's had the lowest 
and Wicomico the highest mean on this index. There were no significant 
differences between elementary and secondary schools on instructional gain. 

Table 21 



Factor 


— ^ 1 


X 


F 


df 


P 


County 


124 


4.00 


2.27 


8/115 


.03 


Caroline 


18 


4.15 








Cecil 


22 


3.79 








Garrett 


9 


4.03 








Harford 


14 


3.90 








Mont{5omery 


5 


4.17 








St. Mary's 


20 


3.69 








Somerset 


9 


3.90 








Washinifton 
Wicomico 


9 
18 


4.32 
4.37 









Schools . As indicated by ratings presented in Table 20, educators 
believed that AT was worth the work. They found In some schools that cross- 
grade articulation improved, and the overall instructional program (in mathe- 
matics) was better organized. Of 249 statements about program benefits, 39% 
related to logistics, and 29% to the overall instructional program, Indicating 
that AT improved instructional organization in adopting schools. 

Of 75 negative comments, 49% related to AT implementation, indicating ^ 
needs for more supplementary materials, reconsideration of time allocations, 
and some reduction of paperwork. In one LEA, classes were divided into two 
groups, and some teachers found instruction difficult. Others were concerned 
about misunderstandings caused by poor communication or insufficient 
inservice. Such misunderstandings contributed to low fidelity of AT use and 
resistance or rivalry in some LEAs. These concerns were felt by school staff 
but were caused in part at the school level and in part at the system level. 
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They related to outcomes in organization, policy, and process — Indicators of 
Institutionalization. 

School organizational outcomes were established fairly well, although 
teachers had a slightly lower perception of SITIP status, fidelity, and 
ownership than did administrators, Ratings assigned indicate the "health" 

of AT. (See Table 22.) 

Policlt.s for management and decision-making were also established fairly 

veil (Table 23), although teachers believed to only a moderate extent that 
decisions were based on data. 

Procedures used to Institutionalize AT indicated that at the school level 
staff were assigned, ln9er^^ice was modified, and resources were allocated , .but 
survey respondents considered that local funds were used to only a slight 
extent (see Table 23).* During Year 3, LEAs continued to receive state 
grants, had to provide matching funds, and were expected to have staff build 
AT into existing tasks (as Indicated by a A.OA mean rating on staff 
accountability) . 

Low mean ratings on Institutionalization indicators, especially when 
accompanied by low ratings on leadership behaviors, suggested project decline.^ 
Such results were apparent in one LEA. However, overall data suggest there 
was good institutionalization at the school level in most LEAs. 

School system . In the four LEAs where the leadership and constructive 
involvement of the central office was Intended, system level outcomes were 
particularly Important. They were less important in the other systems where 
schools took greater responsibility for AT projects. 

Of 51 positive comments about system outcomes, 35% related to logistics, 
20% to process (such as training or participatory planning), 16% to the 
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* However, "in-kind" contributions, such as person time, were considerable, 
especially in LEAs with district-wide strategies. 
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Table 22 

Organizational Outcomes: Active Teaching", 1983-8A 



1 


School 


System 




N 


Mean 


N, 


Mean 


Cognitive 










Status of SHIP established 


109 


3.80 


19 


A. 16 


Close to 100% of teachers asked 
to participate do so regularly . 


107 


3.81 


19 


' 3.95 


Affective 










Local educators feel "ownership" 
of SITIP 


105 


3.35 


19 


3.74 


There is harmony between teachers 
and school-based administrators 
about SITIP 


108 


3.95 


19 


3.68 


There is harmony between school-^ 
based and central office staff 
about SITIP 


103 


3.56 


19 


A. 00 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent). 



Table 23 



Policy Outcomes: Active Teaching, 1983-8A 



Policies 


School ,. 


Sy 


stem 


N 


Mean 


N 


Miean 


Management is shared. 
Decisions are data based. 


22 
107 


3.82 
3.37 


19 

17 


3.8A 
3.63 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent). 
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Instructional program, and 14% to organizational affect (s»^h as team pride, 
mutual respect across, role groups). Only 15 negative ^mments were made, 
mostly relating to process and organizational affect. The sense of both 
positive and negative comme'hts was similar to that given for schools. 

Institutionalization indicator ratings for school systems were almost all 
higher than those assigned to schools.* Organizational outcomes were apparent 
to a fairly large extent (Table 22^, with a clear common understanding of 
SITIP and its use. Policy outcomes were good (Table 23). Examples of 
data-based decisions made district-wide included: extension of allocated time 
for elementary mathematics in one LEA from 45 minutes to 60 minutes daily; 
expansion of the number of teachers involved (to tripled numbers in two LEAs, 
and to more than 200 additional teachers iri a third LEA). In all cases, ^ 
decisions ^te inf lu«iced by student achievement data and teachers assessment 
of the vaijie of AT. Procedural outcomes were also strong, with Inservice 
receiving the highest mean rating. (See Table 24). 



Table 24 

Procedural Outcomes: Active Teaching, 1983-84 



Procedures 


School 


System 


* 


N 


Mean 


N 


Mean 


Inservice modified to support SITIP 


106 


3.92 


20 


4.15 


Staff assignments and accountabili- 
ties modified 


112 


4.04 


19 


3.68 


Resources allocated anpually 


109 


3.28 


19 


3.63 


Local funds used 


18 


2.33 


18 


3.11 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent). 



^ , 

* Teachers rated only schools. Over three years, teachers assigned 

lower ratings to almost all survey items than did administrators. 
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Ratings specific to each LEA demonstrated that lower levels of institu- 
tionalization at the system level put a greater burden on schools. 

Overall, systemwide institutionalization was fairly strongly indicated 
for four LEAs, two of which had been involved for three years, and two for two 
years, all having involved between 75% and 100% of their schools. One other 
LEA, with three years experience, appeared to be moving toward more widespread 
use (with 26% of its school^ involved) , but aystemwide institutionalization 
was weak since implementation was not centrally coordinated, and "press" was 
relatively low. 

Relationships among outcomes . Data were grouped into the following 
Indices: instructional gain, system institutionalization, school institution- 
alization, central office support, and school administrator support. 
Correlational analyses were conducted to determine the relationships among 

r 

these Indices. Correlations among the five indices showed strong direct 
relationships (r - .50) between system institutionalization and two other 
Indices ~ school instltution^Jfczation and central office support. There also 
was a strong positive relationship between school institutionalization and 
school administrator support. (See Table 25.) 

Table 25 

Intercorrelations Among the Indices: Active Teaching 



Index 


1 


2 


3 


4 


5 


1 . 


Instructional Gain 






.23 


N.S. 


N.S. 


2. 


System Institutionalization 




.86 


.77 


N.S. 


3. 


School Institutionalization 






.49 


.63 


A. 


Central Office Support 








.42 


5. 


School Administrator Support 











N.S. = Not significant at the .05 level. 
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Tntluences and Plans 

Influences on past implementation and future action included environ- 
mental turbulence such as: (1) staff changes resulting in shifts In SITIP 
leadership; (2) program changes resulting In revising priorities or 
restructuring tasks; and/or (3) organizational changes. They also included 
the nature and extent of outside (MSDE) assistance, and local perceptions of 
program success. 

Almost all LEAs experienced funding cuts resulting in organizational and 
staff changes, with serious negative outcomes in two cases where loss of 
leadership reduc'ed training opportunities and interest in the project. One of 
those two counties also experienced shifts in program priority at the school 
level. Program changes also occurred in two other counties, causing SITIP 
leaders to work harder to maintain AT but not resulting in project decline. 
Projects survived turbulence when outside assistance "filled the gaps," or 
when local perceptions of success were strong enough to stimulate the extra w 
effort needed to get the work done. Perceptions of success had greater impact 
when a larg^e percentage of a district's schools was involved and senior 
administrators were impressed with the program. In cases where projects were 
primarily school-based, and few schools were involved, envinonmental 
turbulence was a much more serious barrier. 

By the end of Year 3, the four LEAs with system-wide approaches had 
involved 81% of their 100 schools. For Year 4, they planned to complete 
Inservice for elementary and middle school mathematics teachers, provide in- 
depth training in specific concepts, or expand AT to language arts. In 
general, they planned to reinforce instltutionalizatioli. The remaining five 
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LEAS had involved 5% of their 251 schools by June 198A. While one planned to 
add two schools in Year A. the others focused on maintenance. Specific 
activities and factors influencing plans are presented in Table 26. Tn 
general, most participants (86. AZ) planned to maintain their current level of 
implementation, primarily because they believed SITIP influenced student 
achievement (39%). and/or improved teacher's instructional skills (19%). 
Expansion was planned within schools, or to other classes or subjects, rather 
than to additional schools or for additional models. Teachers' and students' 
reactions were influential. Support will probably be reduced, and where it is 
provided will be related to improvement of teaching skills. Plans for 
reduction were minimal, influenced by funding cuts. Four factors that might 
have had a negative impact (listed on the survey) were not considered relevant 
by any respondent. 

Summary and Conclusions 

Over th2 three years of implementation. LEA involvement increased from 
five to nine counties, with impressive increases in the number of schools and 
teacher^ involved (1710 in June 198A). Most teachers using AT for mathematics 
did so for all cUsses in that subject; teachers in other subjects were more 
selective. J'ideli:y was high and concepts of the model well understood, 
instruction in AT mathematics classes observed was excellent, with teachers 
finding AT a helpful framework for lesson organization. 

Following some initial resistance, teachers became more comfortable with 
AT. and Improved their Instructional skills. School-based administrators 
encouraged application, and some used AT to focus attention on mathematics and 
improve cross-grade articulation of the curriculum. Central office staff in 
LEAs with a school-focused implementation strategy provided logistical support 
but were not strongly Involved. In the four LEAs with a syatemwide approach, 
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Table 26 



Activities Planned and Influences Perceived 
by Active Teaching Implementers (N-^US) 



Type of Activities 


% Respondents**! 




Respondents 


Per Factor* 






1 


2 


3 


A 


5 


6 


Maintenance 

name a in current xevex 


86 .A 


39.0 


7.8 


10. A 


19.1 


8.7 




Allow voluntary use 


11.9 


3. A 


.8 


2.5 


3. A 






Expansion 

hxpano ciasoest ouujculs 


33.9 


6.8 


.8 


10.3 


15. A 






ExDand schools 


6.8 


.8 


.8 


.8 


2.5 






fc 

Add another component 

modpl 


3. A 


1.7 


.8 




.8 






Add a new model 


A. 2 


1.7 


.8 






.8 




Support 

Conduct Inservlce 


12.7 


.8 


2.5 




8.5 


.8 




Provide resources 


16.9 


A. 3 


2.5 


.8 


7.7 


.8 




Reduction 

Discontinue SITIP 
r 


3. A 












.8 


Reduce activity by 
25% or more 


1.7 












1.7 


Totals 





58.5 


16.8 


2A.Q 


57. A 


11.1 


2.5 



* Factors: 1. 

2. 
3. 
A. 
5. 
6. 
7. 
8. 
9. 
10. 

** Many reepondents checked more than one type of plan. 



High student achievement data indicate SITIP value. 
SITIP helps achieve a local priority. 
Students and staff like SITIP. 

Teachers' instructional skills improve with SITIP. 
Senior administrators advocate SITIP. 
Funding cuts: other things take priority over JiTIP. 
SITIP is not cost-effective 
There is little staff support for SITIP. 
Senior administrators have little interest in SITIP. 
SITIP has very little to do with local priorities. 
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central office staff coordinated the program and provided various kinds of 
assistance. When administrators and supervisors looked for use of AT during 
their regular classroom visits, there was a press for Implementation which 
stimulated AT fidelity and quality of instruction. When it was also clear 
that administrators and supervisors were helpfully responsive to taachers' 
needs for help, organizational affect also improved. In contrast, when a more 
laissez-faire attitude was apparent in central office staffs attitude to AT, 
the message understood by teachers was that they were still primarily 
accountable for other (established) instructional tasks: the press was low, 
central office support weaker, and application more individualistic. Quality 
did not necessarily differ, but Intensity and consistent use was stronger 
where teachers believed they were expected to use AT. 

There was a clear distinction between school-based and system-wide 
approaches, with the former only moderately successful in moving toward 
institutionalization, even within the school, and the latter establishing 
routine use well. The one site with a capacity-building approach was 
essentially school-based since each school acted autonomously, but central 
office facilitated the knowledge building beyond the initial pilot school. In 
that school, AT was fairly well established, but In other schools in that LEA. 
institutionalization was not yet strongly indicated. 

LEAS planning to continue using AT may wish to review the indicators and 
findings discussed In this report and monitor their progress accordingly, 
applying techniques and strategies to make improvements.* Expectation of use 
by administrators and supervisors in their regular interactions with teachers, 
coupled with specific assistance (e.g., through inservice) and a sense of 



* Each LEA may also review its own "profile" in a separate report: 
SITIP^Case Studies 1984 . 
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8hared responsibility acioss role groups will facilitate implementation. In 
mathematics, continuation of training and assistance, plus additional 
networking or inter-LEA visits t(? exchange specific curriculum and instruction 
ideas (e.g., the videotapes developed by MSDE and some LEAs), will help 
maintain interest and meet some identified needs. In the past, AT 
iraplemenrers and MSDE TAs have generally-^sessed their successes and 
challenges well, and have addressed needs appropriately. It is hoped that 
trend will continue, with realistic recognition that (1) MSDE support will 
gradually be reduced, (2) LEAs with indications of project decline might 
choose to terminate, (3) school-focused sites might independently 
institutionalize at that level, and (4) districtwide use might continue to 
need extra effort from administrators and supervisors. 

Since six of the nine AT LEAs have successfully dealt with environmental 
turbulence, it is expected they will institutionalize, but probably expand 
very little. It is too early to forecast institutionalization for the two 
LEAs which have used AT for one year, but data to date suggest a need for a 
stronger knowledge of AT and of successful implementation practices. One LEA 
has experienced turbulence, but did not resolve the resulting problems. If 
AT is to continue in that LEA, issues of procedure, organization, and 
leadership will need to be explored through data-based participatory decision 
making, to determine redesign or termination. 

In general, AT implementers experienced success in bringing about 
instructional gain. They believed that students achieved more and teachers' 
instructional skills improved. Host intended to continue to use AT for the 
1984-85 year and beyond. Some planned consolidation, reinforcement r or 
expansion. One considered termination of the project, with voluntary 
independent use by individual teachers. Plans for 1985 were influenced by 
successes, by interactive support and administrative leadership, and by 
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environmental turbulence. Apart from the model itself (which has proven 
valuable in Maryland), strong positive influences continued to be those 
Identified in earlier phases of this study (relating to interactive support, 
administrative leadership, and training ard follow-up assistance from MSDE) , 
which contributed to the outcomes of organization, policy, and procedures, 
which in turn indicated extent of institutionalization. 

Mastery Learning (ML) 

This section describes the local implementation of ML. The information 
presented is based on various sources including survey questionnaires 
completed by 98 local implementers (13 central office staff, lA school 
administrators, and 71 teachers). All ML LEAs were represented. 
Approximately A2% of the responding teachers used ML in elementary schools. 
Discussion focuses on: planning; the scope, intensity, and fidelity of use; 
the roles and responsibilities of implementers; the outcomes in terms of 
impact on students, teachers, schools, and school systems; and influences and 
plana for Year A. The section summary reviews the three yea.-s of 
implementation and presents some recommendations for the future. 

Planning 

As stated earlier, ML is a complex model that includes curriculum align- 
ment, diagnostic/prescriptive instruction, and pre-designated mastery levels 
of achievement for the majority of students. Although ML was most often used 
In academic subjects, educators also used the model in a variety of subject 
areas. 

An analysis of local piano for the 1983-8A school year identified LEA 
objectives at the beginning of the year (September 1983), and the extent to 
which project coordinators considered the«e objectives to be achieved at that 
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time and also at the end of the year (June 198A). (See Table 27). In each 
case, the percent of LEAs that "hoped for," "partly achlevrd," or"achieved" 
each objective is indicated. There were nine objectives identified, all 
partly achieved by at least 25% of the LEAs at the beginning of the year 
(having been addressed in Years 1 and 2), and some maintaining that status In 
June (especially in LEAs expanding projects to additional classes or schools). 
Greatest progress was made in curriculum alignment and classroom organization 
(items #6 and #7), and in general there was a shift toward achievement of all 
objectives. Although overall achievement was attained in specific sites or 
subjects, expansion activities were such that systerawide achievement did not 
occur. 

Scope, Intensity, and Fidelity of Use 

Scope of implementation indicates the number of districts, s-chcols, 
teachers, and so forth that were involved. Intensity relates to the amount of 
time a given teacher or class used ML. Fidelity is the extent to which 
teachers Implemented the components of the model as intended by the developer. 
Scope, Intensity, and fidelity in the classroom were influenced by administrators' 
investment of time, and the kinds of activities they carried out. Influenced 
by the implementation strategy employed, these .dimensions indicate the nature 
and extent of use. Table 28 summarizes the use or ML for Year 3. 

Scope . Eight LEAs implemented ML. Four used a lighthouse school 
strategy, and four used a pilot/district strategy (three of which began as 
lighthouse sites but expended in Year 3). Washington was a "new" site, 
attracted by other LEAs' successes, adding ML and AT to their original 
adoption of Student Team Learning. During the summer of 1983, four LEAs 



* In addition to the 28 implementing schools in Baltimore City, 12 other 
secondary schools received inservice during Year 3. 
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Table 27 

Status of Local Objectives, 1983-84: Mastery Learning 







Percent of Sites Achieving Local Objectives 




Local Objectives 




Pre-(Sept. 


1982) 




Post- 


-(June 


1983) 








Status of Achievement* 




Status of Achievement* 






N 


1 ' 


2 


3 


N 


1 


2 


3 


1 


Improve student achievement 
(basic skills) 


8 


37 


63 


0 


7 


14 


43 


43 


2. 


Inprove student achlskyeoent 
(other subjects) ' ^ 


6 


IT A 

50 


50 


0 


5. 


40 


40 


20 


3. 


Inform local educators about model 


8 


37 


25 


37 


7 


14 


43 


43 




irdiQ ftuucocors co U86 moaei 


Q 

O 


37 


50 


13 


c 
0 


17 


50 


33 


5. 


Improve teachers* classroom 


8 


25 . 


75 


0 


0 


17 


67 


17 


o • 


ensure maccn or xnscrucLiouf 
currlculum» and test(s) 


8 


25 


50 


25 


6 


0 


50 


50 


7. 


Help teachers become better 
organized 


8 


25 


63 


12 


6 


0 


83 


17 


8. 


Improve tlme-^on-task 


8 


37 


63 


0 


5 


20 


80 


0 


9. 


Improve students* Involvement In 
learning (motivation) 


7 


29 


57 


14 


6 


17 


50 


33 



* 1 - Hoped for 

2 - Partly achieved 

3 - Achieved 



Note. Total number of LEAs Implementing Mastery 
Learning equals 8. N equals the number of 
LEAs addressing the objectives. 



No data on the status of objectives in June 1984 are available for Washington County. 



116 



c 



Table 28 

Scope of Implementation, Se?t)tember 1983 and June 1984: Mastery Learning 



LEA 


Strategy 


Dimensions 


#of Schools 


Type 


#of Teachers 


//of Students 


oUDj ecc 


Areas 


Sept 831 


June 84 


Sept 831 


June 84 


Sept 83 


June 84 


Sept 83 


June 84 


Sept 83 


June 84 


Allegany 


LS 


2 


2 


0 


0 


17 


18 


260 


350 


D /T A M 
K/ LiA,n, 

Sc,SS,0 


P /T A M 

Sc,SS,0 


Anne Arundel 


LS 


1 


1 


H 


H 


2 


5 


138 


300 


Sc 


Sc 


Baltimore City 


PD 


5 


28 


J/M,H 


J/M,H 


160 


606 


3,635 


22,594 


R/LA,M, 
Sc,SS,0 


R/IA,M, 
Sc,SS;0 


Baltimore County 


LS/PD 


6 


6 


E 


E 


31 


32 


1 ,032 


1,09A 


M 


M 


Carroll 


LS/PD 


5 


c 
5 


T /M 

J/M 


T /M 

J /n 


7 


7 


700 


700 


SS 


^ SS 


Howard 


LS/PD 


2 


6 


J/M 


E,J/M 


3 


35 


90 


1,500 


R/LA,Sc 


R/LA,M, 
Sc,SS 


Washington 


LS 


1 


No data 


E 


No data 


3 


No data 


60 


No data 


M 


No data 


Worcester 


LS 


1 


1 


E 


E 


7 


8 


129 


240 


M 


M 



Subject Areas ; 

IIV 



R /LA-Re ad Ing/ language arts Strategy ; 

M"Mathematic8 
Sc»Science 
SS-Social Studies 

0-0 t her 



LS-Lighthouse school Type ; E-Elementary school 
PD-Pilot district J/M-Junior high/middle 

DW-District-wide H-High school 

CB-Capacity building 0-Other (K-12 and voc. ed.) 
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expanded to additional schools, and two of these LEAs added schools during the 
year (the greatest expansion occurring in Baltimore City).* From 78 teachers 
at the end of Year 1, projects expanded to involve about 715 teachers by June 
1984, with numbers mor^ than tripling (from 230 in September 1983) in Year 3. 
Over 20,000 more students became involved in Year 3, bringing the total to 
approximately 26,800, most of whom were in secondary schools. This expansion 
was greatly facilitated by veteran teachers (trained by MSDE and 
developers in 1981) who functioned as instructional leaders, conducting 
training and coaching for new teachers. Across the state, ML was the second 
most widely used model, used xn 272 of the SITIP schools by 26% of the 
teachers. O'' ^he 49 schools, one was K-12, one vocational-technical, 37 were 
secondary, and 10 were elementary. 

Intensity . Six LEAs were involved since SITIP began, one became involved 
in Year 2; and another oae in Year 3. Some teachers and administrators 
maintained involvement over the three years, but averages were reduced by the 
large number of "expansion" teachers. The mean number of years of teachers' 
involvement was 1.8 (2.07 years for school administrators, and 2.77 for 
central office staff). During the 1983-84 school year, teachers used ML 7.38 
months, on average, with the lowest LEA mean 1.00 month (in Washington — the 
new site). In three LEAs, one or two teachers used ML less than in pre.ious 
years, but in most cases teachers used the model more. In three LEAs ML was 
used for all instruction in a given subject at given grade level" (elementary 
mathematics, high school biology). In one LEA, some SITIP teachers used ML 
for most basic skills classes, while others used ML more selectively. In the 
other LEAs, each participating teacher used ML for at least one course or unit 
of Instruction. The overall in-class time in which teachers used ML was 43%, 
ranging from a mean of 14% for elementary teachers in one LEA to a mean of 55% 
for secondary teachers in another LEA. Consistent use of ML (with fidelity) 
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for a given subject had a positive impact on student achievement. 

Fidelity . As designed, ML requires implementation of 10 components: (I) 
specified objectives. (2) component skills identified. (3) curricula matched 
to objectives. (4) instruction matched to curricula and objectives. (5) tests 
matched to objectives. (6) tests address higher and lower order thinking 
skills. (7) "no fault" formatfve tests given. (8) "correctives" and "enrich- 
ment" activities given after formative tests. (9) summative tests given after 
each unit, and (10) records kept per class/student/objective. In most cases 
mastery was defined as a minimum of 80% students achieving 80% success on 
summative tests. Of the 68 teachers responding to the General Survey. 62% 
carried out all 10 components regularly. All teachers in two LEAs carried out 
all components. Record keeping and inclusion of test items. in higher order 
thinking skills were the components most often neglected (with the latter 
applied by only B2X of the teachers). Fidelity was influenced by the amount 
of time and expertise teachers had for development and planning, and the 
subject or unit addressed. (Where ML was used for basic elementary mathe- . 
matics. higher order thinking skills were more likely to be disregarded t) In 
general the basic concepts of aligned curriculum, and testing and retestlng 
after increased opportunity to learn, were addressed in all LEAs. 

Administrative investment . The amount of time invested in SITIF by an 
administrator during Year 3 averaged about 27 days, with school administrators 
spending more time (mean-31.75 days) than central office staff (21 days). The 
former group included department heads and curriculum coordinators, as well as 
principals and vice principals who -*ere more active In inservice and support 
in many sites than were central office staff. The latter did conduct 
inservice at some sites, but in general spent most SITIP time on 
administration and communication. Activity areas, in order of priority 
allocation of time by administrators, included: (1) inservice. (2) general 
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support,, and (3) cotmnunication/administration.* No administrator stated that 
greatest time was spent on monitoring/evaluation or dissemination/expansion 
activities. 

In general, the quality of implementation was good. The most serious 
weakness occurred (at tcme sites) for "correctives" and "enrichment," with (at 
worst) the former simply a rote repetition of initial instruction and the 
latter "busy work." The' greatest strength was the development of ' curriculum 
aligned with object Iveg^^nd tests, much of which wa? Impressive In quality and 
quantity, and provided evidence of prof esslonal growth for participating ^ 
teachers* 

Roles and Responsibilities 

The SITIP design encouraged the involvement of cross-hierarchical teams. 
However, with the exception of Baltimore City, all LEAs began the program 
using a lighthouse strategy, which meant that central office staff considered 
the program to be school centered and did not intend to be heavily involved.' 
While school teams understood the strategy, a heavy burden was placed on 
teachers in their lirst year, since ML is complex and requires a considerable 
investment A development. By the second year, successes attracted attention, 
and in all but one LEA, expansion plans involved central office staff. ' By the 
end of Year 2, it was apparent that (1) teachers involved in MSDE training 
activities sometimes became instructional leaders, and usually were most * 
active in materials development and on-site coaching for other teachers; (2), 
school-based administrators were ML advocates and active supporters of 
teachers' efforts; and (3) only when central office staff were actively 
. involved through coordination and advocacy based on a thorough understanding 
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* See Roles and Responsibilities for a more detailed discussion of activities. 



of the demands and rewards of ML In an LEA was it possible for the project to 
expand to other sites. Project success was influenced by adequacy of support 
(e.g., teacher t^jne for development), and "messages" of implementation,* and 
could be threatened by environmental turbulence. 

* 

In Year 3, project staff needed to correct errors, consolidate successes, 
and make data-based decisions about use of the model as state funds were 
withdrawn. Local Staff were encouraged to apply the recommendations made in 
earlier SITIP reports, with particular attention to central office staff 
involvement and consideration of support to teachers. The discussion below 
describes role gto\ip activities more specifically in terms of interactive 
support and leadership behaviors. 

Interactive support . Support among LEA participants included exchanging 
information and materials, providing formal training, coaching and trouble- 
shooting, managing logistical arrangemients, and recognizing successes. 
Support from MSDE consisted primarily of training and tec^inical assistance. 
The ML developer maintained communication with MSDE TAs,,but interacted 
directly with only one LEA (at a national workshop) In Year 3* Survey 
respondents rated support received (on a five point ^cale fronl 1.00 « very 
poor, to 5.00 - excellent), and mean ratings assigned by each ML role group 
are presented in Table 29. Overall ratings ranged from a low of 3.15 (for 
developers) to a high of 4.07 (for teachers). Among LEA role groups, teachers 
received very good ratings from all, and (as in previous years) assigned lower 
ratings than did other role groups. 

Ratings within each LEA Indicated the following. ' 



* "Messages" of implementation included overt statements of enthusiasm, 
acknowledgement of teachers' successes and administrators' Investments, as 
well as more subtle notions such as s-incerity in considering points of view, 
building shared trust, and developing shared high standards for ML materials 
and classroom instruction. 
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M.«n r.tlnga range fro« a low of 1.00 (poor) to a high of 5.00 {.xc.ll.nt). 
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• • In all LEAS, teachers were awarded well above average ratings. In five 
cases being rated higher than other role groups, and often rated by 
administrative and supervisory staff as "excellent. 

• School-based administrators were awarded average or above ratings by 
all role groups in all LEAs. 

• Central office staff in all but one LEA (Howard) were awarded average 
or above ratings, and in three cases received the highest ratings of 
all role groups. 

• Low ratings indicated low Interaction and respondents' perceptions of 
relative disinterest on the part of a given role group. 

Ih comparison to previous years, mean ratings across LEAs were about the 

same for all role groups, with a very slight increase for central of f ice staff 

and decreases for the other role groups. These ratings support other data 

which indicated increased effort from central office staff in severfl LEAs 

and/or careful decision-making to meet teachers' needs. 

Administrative leadership . Behaviors found to be important influences on 
project implementation and institutionalization specifically defined, and 
participants asked to assign ratings on each behavior for central office staff 
and school-based administrators. (See Table 30.) 

Affective leadership behaviors were evident, with administrative and super- 
visory 8t»ff advocating the value of ML. demonstrating commitment, interest, 
and enthusiasm, and recognizing teachers' success. Ratings were good for both 
the groups. 

Logistical leadership behaviors were evident foi both role groups, with 
organizational process ratings higher than those related to "press." The 
relatively high ratings for assistance indicated provision of information, 
materials, and training, and effective responses to requests for help. Since 
this administrative behavior (assistance) was strongly recommended by MSDE 
(and it influenced teachers' success), these ratings were positive. School 
and district comn&inication ratings were good, reflecting administrative and 
supervisory staff efforts to correct some confusion that had been apparent at 
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Table 30 

Administrative Leadership Behaviors: 
Mastery Learning, 1983-84 





Mean Ratings Assigned 


oenAVXOro 


To Central 
ui r ice bcai I 
(N-81) 


To School-Based 

Auaixnxo UX.OUUL o 

(N-83) 


Aiiective 






Demonstrate commitment 


4,35 


^ . 1 / 


Provide support 


H,00 


J • Vo 


Logistical 






Press for fidelity 


2.64 


2.72 


Press for Intensity 


2.69 


2.80 


Provide assistance 


4.16 


4.09 


Coordinate LEA communication 


3.76 


3.40 


Coordinate school communication 


3.61 


3.55 


Implement data-based decision-making 


4.04 


3.71 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent). 
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some sites in previous years. Data-based decision-making, particularly by 
central office staff, received good ratings (better than for the other three 
models) which was commendable. 

In three LEAs teachers undertook leadership tasks. In Anne Arundel the 
department head took charge of the project, directly affecting the five 
teachers in his department. In Baltimore City, a teacher from the initial 
pilot school was released from classroom responsibilities to coordinate the 
project, directly involving 28 secondary schools. In Baltimore County the 
pilot school teachers participated in cross-hierarchical planning, and 
conducted training and coaching for 32 teachers in six schools. In all cases 
teachers as instructional leaden demonstrated considerable expertise in ML, 
were effective trainers and helpful coaches. They exhibited the administra- 
tive leadership behaviors listed in Table 30 Learning from experience and 
various SITIP activities, their "press" for quality implementation reflected a 
colleagial understanding of the demands of ML. As one said, "This year I've 
learned tolerance." They acknowledged their own professional growth and were 
capable in transferring their knowledge to others, taking into account the 
varying levels of interest and ability of partidipating teachers. 

Overall, interactive support and leadership for ML were good. However, 
in three LEAs, central office staff could have attended more to school-based 
concerns and, in one case particularly, to recommendations from MSDE about 
project management. Where central office involvement was high, communication, 
fidelity, and scope of implementation were good. School based administrators 
were supportive and understood how ML could help them achieve school goals. 
Teachers continued to work hard, modifying development efforts in some sites. 

Outcomes 

ML had an impact on students, teachers, schools, and school systems. 
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Participants' perceptions were supported by data summaries, locally developed 
classroom and inaervice materials, and newspaper articles. For Year 3 of this 
study, greatest attention was given to outcomes relating to institutionaliza- 
tion. 

Students . Survey respondents agreed that students benefited from ML: 
they enjoyed it (mean rating of 4.13) and increased achiev^^ment (4.04). (See 
Table 31.) The goal of having at least 80% of the students achieve at least 
80% mastery of core curriculum was achieved in most classes. However, careful 
analysis by several LEA indicated that the definition of mastery was revised 
somewhat after initial use of an ML course. The following example (from 
Carroll) is an illustration. 

. On a pretest, 10% of below average students achieved scores of 80% or 
better, and 12.5% of above average students achieved that score (for 
an average of 6% student mastery) . 

• Results of course unit tests were 85.7% of above average students 
achieving mastery, 79% o. average students achieving mastery, and 44% 
of below average students achieving mastery. 

• On the post test, 44% of below average students achieved mastery, and 
80.5% of above average students achieved mastery (for an average of 
60% student mastery). 

• All students took identical tests and were taught from the same set of 
lesson plans. Mastery level was revised downward. 

in another county, mastery was initially defined as 100% students of the 
achieving 80% success, but the amount of time for correctives was such that 
levels were revised to 80-80. In some classes an 80-80 mastery was strictly 
maintained, but only for the core curriculum. In some schools, mastery 
levels varied, and even greater differences occurred between classes using 
tests of different difficulty levels. However, all participants agreed that 
student achievement improved with ML. 

In another county (Allegany), vocational-technical students, using 
criterion-referenced tests, achieved 100% mastery in cosmetology and 86% 
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achieved 80% mastery in reading; on average, 84.5% of students In grades K-6 
achieved 80% mastery In mathematics, and between 80.1% and 93% of students In 
grades 7-12 achieved 80% mastery In various subjects. Corrective Instruction 
facilitated mastery (after formative testing) for as many as 46% of the 
students. 

Data summaries based on CAT scores were submitted by Worcester and on 
41e£r^peli t3 R a cor aa-by-B»lt"laege^€ountyT both illustrating gains in elementary 
mathematics, the latter comparing ML with non-HL students in similar schools 
using pre and post test scores. In Worcester, ML students in first and second 
grades "grew** between 11 months add 13 months over the seven month school 
year. In Baltimore County, ML students gained an average of 1.7 years of 
grade equivalent compared with an average of 1.2 years for non-ML students. 

Student attitude data from Carroll and Allegany indicated that at least 
75% believed they learned more and got better grades with ML. They also found 
the lessons easier » and understood and enjoyed them. 

Overall » ML had a strong positive lnq>act on students In all LEAs. The 
expectations of success were evidenced In survey respondents* statements » such 
as» "I am seeing students who are achieving more than I thought possible." 

Teachers . Teachers' knowledge of effective teaching and skill In 
Instruction Improved. Specific gains for teachers Included: an Increased 
sense of professionalism, better planning and organization of Instruction, 
improved ability In diagnostic/prescriptive teaching and In curriculum 
development. While most teachers enjoyed KL, there were some unresolved needs 
Including: time to develop new units of Instruction; help In developing multi- 
choice tests that ^ cover both high and low order thinking skills, and In 
developing effective corrective and enrichment strategies; and ways to reduce 
time Invested In record-keeping. In many cases, teachers transferred some ML 
concepts to other classes and subjects. 
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Impact on students and teachers combined was defined as Instructional 
gain. There were no significant differences among the LEAs or between 
elementary and secondary schools on Instructional gain. 

Schools . As Indicated In Table 31, educators agreed that ML was worth 
the work It took. Of the 104 statements about program benefits, 37% related 
to logistics, 23% to the overall Instructional process, and 21% to the 
positive Impact on students. Indicating that ML Improved the management and 
planning of curriculum and instruction. 

Of the 66 negative comments, 70% related to the overall Instructional 
process, focusing primarily on time demands and concerns identified by 

teachers. ^ 

School organizational outcomes were good. (See Table 32.) Ratings 
assigned indicated the "health" of ML. In six of the eight LEAs, there was 
little disagreement among role groups about the cognitive and affective 
outcomes at the school level, and little difference among those counties on 
the five dimensions examined, although "regular participation" ratings did 
vary somewhat among districts. Results for the other two LEAs (Washington and 
Howard) indicated strong differences of perception between tea' hers and 
administrators at one site that began ML in 1983, and low mean ratings for 
both districts on cognitive and affective outcomes. In both LEAs, leadership 
changes may well have Influenced these outcomes. 

Ratings for policies of management and decision-making again indicated 
appropriate action for six LEAs, but for two LEAs, management was not shared 
across role groups, and in one case, decisions were not data-based. Overall 
means were fairly good. (See Table 33.) 

Procedural outcomes followed a similar pattern, vlth the same two LEAs 
indicating lower ratings than the others, particularly for use of local funds, 
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Tab It 31 

Instruct lonal Impact as Perceived by 



Impact on Instruction 



itmctlonal Value 



Works in classrooa 

Is worth the work it takes 

impact on Teachers 

Teachers enjoy It 

Teachers have increased knowledge 

Teachers have increased skills 

Impact on Students 

Students enjoy it 

Students' achievement has increased 
Students are learning and retaining more 
Students' general behavior is better 
Students are taking more responsibility 
for their own learning 



Role Groups 




CO 
N-13 


SA 
N-14 


T 

N-70 


Total 
N-97 


4.38 
4 .00 


4.00 
3.64 


4.41 
4.14 


4.35 
4.05 


3.77 
.4.69 
4.54 


3.93 
4.14 
3.93 


4.09 
4.17 
4.17 


4.02 
4.24 

A 1 Q 


4.08 
4.15 
4.00 
3.54 


3.86 
3.71 
3.71 
3.50 


4.20 
4.09 
3.76 
3.56 


4.13 
4.04 
3.79. 
3.55 


1 3.85 


3.57 


3.75 


3.73 



H..n r.ttngi r.ng. fro. 1.00 (.trongly <ll.«gr..) to 5.00 (ttrongly .gr..) 
- "otrll Oftlc.; SA - School Ad-lnl.tr.tor.i T - T..ch.r. 
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Organizational Outcomes: Mastery Learning, 1983-1 



Oil ^ crwii^ 0 

V/U U ^V^IUC D 


School 


Sj 


rstem 




N 


Mean 


N 


Mean 


Cognitive 


* 








Status of SITIP established 


89 


3.85 


19 


3.74 


Close to 100% of teachers asked 
to participate do so reguiarxy 


85 


.3.85 


19 


3.47 


Affective 










Local educators feel "ovmership" 
of SITIP 


86 


3.35 


18 


3.28 


There is harmony between teachers 
and school-based administrators 
about SITIP 


87 


3.84 


18 


4.17 


There is harmony between school- 
based and central office staff 
about SITIP 


85 


3.94 


19 


4.21 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extant). 



Table 33 

Policy Outcomes: Mastery Learning. 1983-84 



Policies 


School 


System 


N 


Mean 


N 


Mean 


Management is shared 


19 


3.79 


20 


3.50 


Decisions are data-based 


87 


3.83 


20 


4.00 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent). 
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and In one case for staff assignments and accountabilities. Overall mean 
ratings at the school level Indicated that procedures to facilitate 
Institutionalization of ML were moderately well established. (See Table 3A.) 

Low mean ratings on Institutionalization Indicators » especially when 
accompanied by low ratings on leadership behaviors » suggested that the project 
was not doing well. In one caset low ratings were not unexpected since that 
LEA was just beginning. In the other LEA» the project may decline If 
appropriate action Is not taken. 

School system . System level results are reported In Tables 32 » 33 » ^nd 
34. They are more Important for LEAs using a pllot/dlstrlct strategy (e.g** 
Baltimore County) than they are for LEAs using a lighthouse school approach 
(e.g.t Anne Arundel). 

In most cases » Institutionalization Indicator ratings for the system 
level were all lower than those assigned to schools. The exceptions related 
to "harmony/' "decision-making/' a^id "Inservlce," with the comparative 
differences on those three dimensions Influenced In part (positively) by the 
real efforts of administrators » particularly In five counties (Alleganyt 
Baltimore Clty» Baltimore County^ Carroll^ and Worcester) » and in. part 
(negatively) by lower ratings assigned by teachers to the school level.* 
Overall, system organizational outcomes were moderate to good# particularly In 
terms of Inter^-organlzatlonal harmony — an Improvement over the first year 
when friction between role groups was strong. Policy outcomes were goodt 
particularly In relation to data-based declslon-*maklng — as evidenced by the 
quality and quantity of program Impact data collected and used by ML projects. 
Procedural outcomes were fairly good. 



* Teachers did not rate system factors. 

lis 
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Table 34 7^' 
Proctdural Outcomes: Mattery Learning, 1983-^4 



Procedures 


School 


S 


^stem 


N 


Mean 


N 


Mean 


Inservlce modified to support SITIP 


82 


3.71 


21 


3.76 


Staff assignments and accountabili- 
ties modified 


86 


3.72 


19 


3.53 


Resources allocated annually 


88 


3.79 


20 


3.70 


Local funds used 


20 


3.15 


18 


3.22 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent). 
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Low system level outcomes put a turden on schools. This was particularly 
Important for the four LEAs using a pilot/district strategy. In three cases, 
system outcomes were good, and schools benefited, with teachers responsive to 
the support provided. However, In one case. Institutionalization is uncertain 
since system outcomes were not strong. 

Positive perceptions of central office staff, influenced by their 
participation in MSDE training events, and by instructional gain data, were 
strongly evident for ML. One adainstrator stated those perceptions: "SITIP 
Is one of the best things MSDE had done for LEAs. It hao provided us with an 
opportunity to do something we Would not have done on our own by bringing^in 

i 

experts, and providing follow-up and an ongoing support system." 

Relationships among outcomes . Data were grouped into the folJ.owing 
indices: instructional gain, system Institutionalization, school 
institutionaliietion, central office support, and school^administrator 
support. Correlational analyses were conducted to determine the relationships 

^ among these Indices. 

^ Correlatlohs among the five indices showed strong direct relationships 

(r - .50) between system institutionalization and three other indices — 
school Institutionalization, instructional gain, and central office support. 
School Institutionalization correlated strongly with central office and school 
administrator support. Central office support was positively related to 
school administtator support. (See Table 35.) 
Influences and Plans 

Influences on past Implementation and future action Included 
environmental turbulence such as: (1) staff changes resulting in shifts in 
SITIP leadership; (2) program changes revising priorities or restructuring 
tasks; and/or (3) organizational changes. They also included the nature and 
extent of outside (MSDE) assistance, and local perceptions of program success. 
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Table 35 

Intercorrelatlons Among the Five Indices: Mastery Learning 



1 

Index 


1 


2 


3 


4 


5 


1. Instructional Gain 




.63 


.34 


.43 


N.S. 


2. System Institutionalization 






.87 


.63 


.45 


3. School Instltltlonallzatlon 






.56 


.55 


4. Central Office Support ^ 








.61 


5. School Administrator Support 











N.S. -Not significant at the .05 level. 

Note ; The number Of cases upon which the correlations were 
calculated varied. v 
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Almost all LEAs experienced finding cuts, and the two largest districts 
(Baltimore City and Baltimore County), both planning ML expansion, bemoaned 
the availability of so Uttle money for such an effective program. The li^ajor 
impact of low funding was slower and more, cautious expansion. » 

Three other kinds of changes occurred: (1) changes in staff resulted^n 
leadership shifts in three LEAs, with one being beneficial and the other two - 
being somewhat negative sinde expertise was lost; (2) changes in prqgrams led 
to considerable ML expansion in three LEAs; and (3) there were changes in 
organizational arrangements in three LEAs as a r^s; • of changes in 

r 

leadership. — 

While all LEAs experienced changes, and six experienced all three kinds 
of changes, negative impact was' relatively low, especially when project 
leadership remained constant and applied thelmowledge exchanged at MSDE- 
sponsored training activities (or at inter-site visits). Turbulence was a 
problem when loss of leadership expertise was combined with low perceptions of 
success in terms of instructional gaiiC and organizational outcomes. This 
occurred in two LEAs. 

By the end of Year 3, the four LEAs using a lighthouse strategy had 
together involved fiv^ of their 189 schools and planned to continue 
implementation at about the same level.* The four pilot/district LEAs had 
Involved* about IIZ of their 404 schools and planned to continue expansion in 
Year 4.* Survey respondents indicated specific activities and influential 
factor8,_^j^i^ later (without reference to survey data) plans were submitted to 
MSDE by project directors. For most LEAs, individual re^onses and 



* One lighthouse site aui one pilot/district site planned extensive training 
(involving the ML develop In one case) with systematic follow-up and 
coaching for elementary, si.i oul^ Irtieietited in implementation. 
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coordinators' plans agreed with each other. However, in one of the LEAs 
negatively Influenced by environmental turbulence, a few survey responses 
recommended termination In light of more pressing priorities, but the project 
plan submitted Indicated expansion. 

Survey responses are summarized In Table 36. In general, most participants 
(61.2%) planned to maintain their current level of Implementation or expand to 
other classes or schools (A7%) primarily because high student; achievement data 
Indicated SITIP value. Provision of support through Inservlce and resource 

i 

allocation vas planned. Very few respondents advocated reduction. In 
addition to student achievement, other strong influences on plans Included: 
iinprovement of teachers' instructional skills (identified by 50% of 
respondents), student and staff enjoyment (28.5%), and achievement of a local 
priority through SITIP (24.4%). Plans for reduction were influenced by 
funding cuts and low staff support. 
Summary and Conclusions 

Over the three years, local involvement in ML increased from six to 50 
schools, from 78 to 715 teachers, with eight LEAs participating in June 1984. 
In the five LEAs involving elementary grades, ML was used primarily in 
mathematics classes; in the five LEAs involving secondary schools, ML was used 
in a wide variety of subjects. Intensity of use increased for most sites (but 
'Was very low for the new LEA), and fidelity was good. The most serious 
weakness of classroom use related to strategies for corrective and enrichment 
activities. The greatest strength was the organization of instruction 
facilitated by the quality of curriculum and test development (aligned with 
core objectives) . 

Materials development and on-site coaching (as well as classroom 
Instruction) were frequently undertaken by teachers, who became instructional 
leaders at several sites. Leadership was also undertaken by school-based 
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Table 36 



Activities Planned and Influences Perceived 
by Mastery Learning loplementers (N-98) 



Type of Activities 



% Respondents** 



1 



X Respondents Per Factor* 



Maintenance 



Maintain current level 
Allow voluntary use 
Expansion 



Expand - classes 

Expand - schools 

Add another component 
of model 

Add a new model 

Support 



Conduct inservice 



Provide resources 



Reduction 



Discontinue SITIP 

Reduce activity by 
25% or more 



61.2 
16.3 

47.0 
19.4 
6.1 

1.0 

29.6 
30.6 

1.0 
3.1 



27.6 
3.1 

23.5 
4.1 
2.0 

1.0 

5.1 
6.1 



Totals 



72.5 



5.1 
2.0 

6.1 
3.1 
1.01 



8.2 
6.1 

6.1 
2.0 



5.1 
2. 



4.1 
2.0 



24.4 



28.5 



10.2 
3.1 

6.1 
4.1 
3.1 



4.1 



11.2 
12.2 



50.0 



2.0 
3.1 



2.0 
5.1 



1.0 



16.3 



1.0 



1.0 



1.0" 



2.0 



1.0 



1.0 



* Ten factors were listed on the survey, of which eight were indicated as influential in 
planning by some respondents. 

1. High student achievement data indicate SITIP value. 

2. SITIP helps achieve a local priority. 

3. Students and staff like SITIP. 

4. Teachers' instrt'.ctional skills improve with SITIP. 

5. Senior administrators advocate SITIP. 

6. Funding cuts: other things take priority over SITIP. 

7. SITIP is not cost-effective 

8. There is little staff support for SITIP. 

9. Senior administrators have little interest in SITIP. 
10. SITIP has very little to do with local priorities. 

** Many respondents checked more than one type of plan. 
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admtnistrators and central office staff, with the latter particularly 
appreciated in three LEAs. Useful leadership behavior included evidence of 
commitment and support accompanied by effectively coordinated communication, 
data-based decision-making, and "press" for fidelity and intensity of 
implementation. The "press" was lower than desirable in several cases, but 
where it occurred (accompanied by assistance), it contributed to quality 
implementation. 

Impact on students and teachers was very good, with empirical data 
supporting educators' perceptions that student achievement increased, with a 
greater number of students achieving more than usually expected. There were 
no significant differences for instructional gain among LEAs or types of 
schools. Organizational, policy, and procedural outcomes were good, although 
the new LEA and one other had low ratings for institutionalization. School 
institutionalization was strongly correlated with support from school and 
central office administrators. District institutionalization was correlated 
with instructional gain, school institutionalization, and central office staff 
support. 

Over the period of implementation, all LEAs experienced some form of 
environmental turbulence, but only two seemed to have been negatively 
influenced. In both cases, organizational outcomes were low or moderate, and 
instructional gain was not clearly determined. Negative influences 
experienced by at least two LEAs during the three years included: (1) 
initial confusion in assistance from MSDE resulting in misunderstanding or 
additional workloads in LEAs; (2) too much work planned too soon; and (3) 
changes in organization or leadship, resulting in loss of expertise or 
conflict of purpose among role groups. Positive influences (experienced by at 
least two LPAs) included: (1) training and follow-up activities by the 
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developer (Block), MSDE. and LEA teams that met participants* stated needs; 
(2) responsiveness of administrators to teachers' concerns; (3) local use of 
the model to accomplish local priorities; (4) stability and expertise of local 
leadership, with press for fidelity and attention to application of 
Improvement strategies; and (5) clear stable purpose, acceptable to teachers. 

In two districts. Institutionalization was somewhat uncertain, but In the 
others Indications were posit Ive^ The quality of implementation was good, and 
results worthwhile. As LEAs move into Year 4, at the classroom level 
attention should be pKid to corrective and enrichment strategies; at the 
school level» to support in re'cord<»keeplng and expansion of units; at the 
district level, to exchange of material and expertise and use of mechanisms 
for "veterans" to assist "new" sites; and between LEAs to learning from each 
other's experience in order to improve implementation overall and ensure 
institutionalization. 

Student Team Learning (STL) 

This section describes the local implementation of STL. The information 
presented hert: is based in various sources, including survey questionnaires 
completed by 53 local Implementers (five central office staff, 10 school 
administrators, and 38 teachers). Six LEAs were represented. Approximately 
32% of the responding teachers used STL ln^«lementary school. Discussion 
focuses on: planning; the scope, intensity, and fidelity of use; the roles 
and responsibilities of Implementers; outcomes in terms of Impact on students, 
teachers, schools, and school systems; and Influences and plans for Year 4. 
The section summary reviews major changes over the three years of Implementa- 
tlon and presents some recommendations for the future. 



Planning 

As stated earlier. STL encourages cooperative learning and Includes three 
different approaches, all of which require students to work In small groups. 
It can be used In a variety of subjects as long as appropriate materials are 
available. LEAs adopting STL considered it an alternative instructional 
process to be used primarily at the discretion of teachers. 

An analysis of local plans for the 1983-84 school year identified LEA 
objectives at the beginning of the year (September 1983). and the extent to 
which project coordinators considered objectives to be achlfeved at that time 
and also at the end of the year (June 1984). (See Table 37.) There were nine 
objectives identified, all partly achieved by at least 25% of the LEAs at the 
beginning of the year (having been addressed in Years 1 and 2). and some 
maintaining that status in June. Progress was made in all areas, and was 
strong in Informing educators, and improving student achievement and 
Involvement. Since policy changes and impleimentation strategies resulted in a 
teacher or school focus of change (rather than a district focus) for all but 
one LEA. systemic achievement of objectives was not evident. 
Scope. Intensity, and Fidelity of Use 

Scope indicates the numbers of districts, schools, teachers, and so forth 
which were involved. Intensity relates to the amount of time a given teacher 
or class used STL. Fidelity is the extent to which teachers implemented the 
components of the model as intended by the developer. Scope, intensity, and 
fidelity in the classroom were influenced by administrators' investment of 
time, and the kinds of activities they carried out. Also influenced by the 
Implementation strategy employed, these dimensions indicate the nature and 
extent of use. Table 38 summarizes the use of STL for Year 3. 

Scope . Nine LEAs implemented STL. iour of which were virtually "on their 
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Table 37 

Status of Local Objectives. 1983-8A: Student Team Learning 



* 1 - Hoped for 

2 - Partly achieved 

3 " Achieved 

No data are available for Prince George's 
Washington County. 





Percent of Sites Achieving Local Objectives 


Local Objectives 




Pre-(Sept. 1982) 




Fo8t-(June 


1983) 




Status of Achievemeat* 




Status of Ach 


.evement* 




N 


!_ 


2__ 


3 


N 


1__ 


2 


3 


1 • mprOVB obuuonb acnxtsveoiciib 

(basic skills) 




57 


43 


0 


6 


17 


50 


33 


2. Improvtt student achievement 
(other subjects) 




7 1 




0 


5 


20 


40 


40 


3. Inform local ' ducators about model 






57 


29 


6 


17 


17 


67 






0 


86 


14 


5 


20 


40 


40 


5. Improv* taachers' classroom 
conips wvucv 




0 


100 


0 


5 


0 


60 


40 


6. Ensure natch of InstructloUf 
curriculum, and test(s) 




25 


25 


50 


5 


0 


40 


60 


7. Help teachers become better 
organized 




0 


80 


20 


5 


0 


60 


40 


8. Improve tlme-on-^task 




20 


80 


0 


4 


0 


50 


50 


9. Improve students' Involvement In 
learning (motivation) 


1 ' 


0 


86 


14 


6 


0 


50 


50 



Note . Total number of LEAs Implementing Student 
Team Learning equals 9. N equals the 
number of LEAs addressing the cbjectlvt. ^ 

and Baltimore counties. No June 1984 data are available for 
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own" for Year 3 since the only support they received was representative 
participation in MSDE follow-ups.* In those four independent LEAs (Baltimore 
County. Prince George's, Washington, and Worcester), teachers who had 
participated in previous years were allowed /encouraged to continue on their 
own: administrative and supervisory (A4S) staff made no formal commitment to 
assist those teachers. State funds were not allocated, to those STL sites: 
Prince George's requested no funds for Years 2 and 3; the remaining 
independent LEAs used state funds to support other SITIP models.** 

While the four independent LEAs initially planned various implementation 
strategies, in Year 3 each had between one and ten schools functioning as 
independent lighthouse sites. For Year 3. while two LEAs could be considered 
pilot/district since they had centralized coordination to Involve several 
schools, two others were lighthouses, and five were capacity builders. In 
practice, all LEAs functioned primarily as systems of loosely coupled schools, 
each of which was fairly isolated (like a lighthouse), and in which teachers 
helped each other. 

As can be seen in Table 38. survey data are incomplete. Estimates based 
on other data sources indicate that about 30 schools were Involved at the 
beginning of Year 3. and about 38 by June 198A, with approximately 200 
teachers participating. While expansion was apparent in two LEAs. the number 
of participating teachers gradually decreased, in other sites. Across the 
state, STL was used in about 1A% of the SITIP schools by about 7% of the 



* Statistical data and survey responses were available for six LEAs: 

Calve-t. Charles. Dorchester. Montgomery. Queen Anne s, and Worcester; and 
they are sources for information presented in tables in this report. 
Information about three of the independent sites (based on interviews and 
LEA presentations at follow-ups) is woven into some of the discussion. 

** Montgomery also implemented < ther models but allocation of funds for STL 
was not changed. 
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T«bl« 38 



Scop, of l«pl««nt.tlon. S«pt«rtT 1983 wtf Jun. 1984: Stu<I«nt T«« U.mlng 




* At pilot .chool ~ ^3 of itudrot. at oth.r ■Iddl. schools. 



^ , , •_»i---n.^rT school Subloct ArMs: R/U-losdlni/lanfusgo *rt» 

Strsfiy : LS-tlghthous. school Hei: *""*?""i5!w/2!;L ^^'^^^ N-IUthsMtlcs 

PD-Pllot district J/M-Junlor hlfh/«iddls 8c-8cl«nc« 

CB-Cspsclty building H-Hlgh school gg-gocUl studios 

DW-Olstrlct-wlds 0«Klthsr 0-Othor 
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teachers. About a third of the schools were elementary; almost all secondary 
schools involved were middle or Junior high schools. 

Intensity . All but one LEA began implementation in Year 1, and many 
teachers and administrators maintained involvement over three years. The 
average for administrative and supervisory staff was two-and-a-half years, 
with leadership reassignments in only one LEA. On average, teachers were 
involved for Just under two years (relatively low means were influenced by 
expansion site teachers). During the 1983-84 school year, teachers used STL 
over a period of 5.03 months on average, with the lowest mean nine weeks (one 
marking period in a high school), and the highest nine months (the full year 
m an elementary school). More than 55% of the teachers used STL more in Year 
3 than they had done previously, Wilth about half of the remaining teachers 
using the model for the same amount of time, and half for less time. On 
average, teachers used STL for about 18% of their in-class time across the 
year, ranging from 12% in an elementary school to 25% in a middle school. 
Many teachers used a particular STL approach (e.g., STAD) for one part of the 
curriculum (e.g., multiplication tables), and another approach later in the 
year for a different subject (e.g., social studies). Other teachers used 
traditional instructional techniques, adding STL for particular lessons or 
units within a course, often using STL for review rather than for instruction 
in new content. No one used STL consistently for a given course or subject. 

Fidelity . STL has three approaches, each of which has key components. 
Regardless of the approach used, essential components ijiclude: (1) materials 
are available for peer tutoring, team practice, and individual and tournament 
quizzes, (2) peer tutoring takes place. (3) each team includes a mix of klrids 
of students (on given criteria). (4) quiz /tournament scores relate to 
individual and team achievement, and (5) successes are publicized. Of the 38 
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teachers responding to the general survey. 59Z carried out all 5 components 
regularly. In two LEAs, 75% of the teachers carried out all components. In 
three LEAs, 50% carried out all components. Team composition was the most 
commonly practiced component. All other components were practiced by at least 
82Z of the teachers.' STAD was the simplest and most popular approach (26% of 
the teachers used It), followed by TGT (22%), then Jigsaw (11%) — the most 
complex approach. 

Fidelity and Intensity were influenced by the amount of time and expertise 
teachers had for development. (Although STL developers offered materials, 
teachers found that they had to be adapted to fit local curriculum, and often 
teachers preferred to develop their own materials.) Another Influence was the 
perception that STL (unlike AT or ML) was not a process to be used contin- 
uously for general Instruction, but rather a "fun technique" to stimulate 
motivation, to break routine, or to be used as an alternative approach for 
* review. This perception Influenced the expectations, or "press," by 
administrative and supervisory staff, and teachers acted on those perceptions 
(rather than on the developers' hope for more pervasive use). At Its lowest 
fidelity, STL was a class organized by four-person groups, receiving whole 
class instruction followed by "practice" learning in pairs (often with one 
partner completing a worksheet while the other watched) . At its highest 
fidelity, students worked in four-person groups using carefully designed 
materials that encouraged peer tutoring/ learning, procedures and expectations 
were obviously well-understood, and the teacher could monitor and assist, as 
needed. 

Administrative investment . The amount of time Invested by administrative 
and supervisory staff during Year 3 averaged 19 days, with school-based 
administrators spending more time (mean-20.20 days) than central office staff 
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(12.50 days). The former group Included department heads and curriculum 
coordinators as well as principals and vice principals who were more active 
than central office staff In general support or Inservlce In all sites. The 
latter were Involved to a small extent In Inservlce in a few sites, but. In 
generajl^ spent most SITIP time on administration andLxommunlcatlqn. Activity 
areas. In order of priority allocation of time by a^lnlstrators , Included: 
(1) Inservlce, (2) general support. (3) communlcatlot^admlnlstratlon, and (A) 
monltorlnf>/evaluatlon.* No administrator said that dissemination/expansion 
activities were the most time-consuming tasks. 

The quality of Implementation was patchy. Differences were apparent^ not 
only between sites but also within schools, with one teacher using her regular 
worksheets and managing tlme-on-task poorly, and another teacher managing 
everything very well. The extreme of poor quality was less common. The 
greatest strength of STL was the focus on students (rather than on subject 
matter), which was always apparent and which was particularly appreciated In 
secondary schools. 
Roles and Responsibilities 

The SITIP design encouraged the Involvement of cross-hierarchical teams. 
By the end of Year 2, It was apparent that: (1) teachers Involved In MSDE 
training activities often became Instructional leaders or "key" teachers who 
coached others in their 'schools; (2) school-based administrators of lighthouse 
sites and others trained by MSDE usually were supportive of their teachers and 
facilitated SITIP implementation; and (3) central office staff trained by MSDE 
were also supportive. For STL, in all LEAs all role groups were Involved, but 



* sea Roles and Responsibilities for a more detailed discussion of activities. 
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central office staff remalued primarily administrative » school administrators 
provided general support, and teachers took primary responsibility for 
implementation. In other words, roles and tasks for STL were basically the 
same as for the regular program. Locally trained educators could become 
advocates and Invest time and energy If certain procedures wer% used to build 
commitment and provide logistical support. However, project success depended 
on very careful coordination of various activities and "messages,"* and could 
be threatened by environmental turbulence. 

In Year 3, project staff needed to correct errors, consolidate successes, 
and make data-based decisions about use of the model as state ^unds were with- 
drawn. Local staff were encouraged to apply the recommendations made in 
earlier SITIP reports, with particular attention to central office staff 
involvement. The discussion below describes role group activities more 
specifically in terms of interactive support and leadership behaviors. 

Interactive support . Support among LEA participants included exchanging 
information and material^, providing formal training, coaching and trouble- 
shooting, managing logistical arrangements, and recognizing successes. 
Support from MSDE and STL developers consisted primarily of tralni»^.g and 
technical assistance. Survey respondents rated the support received (on a 
five point scale from 1.00-very poor to 5.00«excellent) , and the mean ratings 
assigned by each role group are presented in Table 39. Overall ratings ranged 
from a low of 3.13 (for developers) to a high of 4.18 (for teachers). Among 
LEA role groups, teachers received good ratings from all, and (as in previous 
years) assigned lower ratings than did other role groups. 

* "Messages" of Implementation included overt statements of enthusiasm for the 
project, acknowledgement of teachers' successes and principals* Investments, 
as well as such more subtle notions as sincerity in considering points of 
view, building shared understanding, and trust about project purpose. 
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• Table 39 

Perceptions of Support Received: Student Team Learning, 1983-84 



Respondents 



Central Office Staff 



School Administrators 



Teachers 



Total 



Teache^rs 



N Means 



5 

10 
34 
49 



4.20 
4.40 
4.12 
A. 18 



School 
Administrators 



N Means 



5 
8 

38 

51 



4.00 



4.25 
3.58 
3.73 



Support Groups 



Central 
Office Staff 



N Means 



5 

10 
34 
49 



4.20 
4.40 
3.38 
3.67 



MSDE 



N Means 



4 

0 

32 
46 



4.75 
4.10 
2.91 
3.33 



DivelopTi 



N 



4 
9 

34 
47 



Ms ant 



4.75 
3.33 
2.88 

3.13 



Mean ratings range from a low of 1.00 (poor) to a- high of 5.00 (excellent). 
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Racings within tach county iXlcatad the following. 

^» In all LEAs, taachers w^re awarded above average ratings, reflecting 
their steady hard work 1^ Implementation. 

e With the exception of one LEA (Mont gome ry , school administrators were 
awarded above average railngs, with scores higher than the li teachers 
In one LEA where they were particularly active (Queen Anne s) . 

• With the exception of Montgomery, central office staff received above 
average ratings. In Dorchester, a pilot/district where central office 
staff were particularly active, mean ratings were 4.58. 

• Low ratings Indicated low Interaction and respondents' perceptions of 
relative disinterest on the part of a given role group. Coupled with 
data from other sources, ratings suggest that energy and Interest 
remained fairly high In five LEAs (Calvert, Charles, Dorchester, Queen 
Anne's, and Worcester) but declined In the four other LEAs where 
Implementation responsibilities fell to Individual teachers.* 

In comparison to previous years, mean ratings across LEAs were somewhat 

lower for all role groups except teachers. This might be expected since 

activities became more routine for veteran implementers. In general, in third 

year projects, energy levels were lower, which reduced interactive support 

among local participants. In Independent sites, administrative and 

supervisory staff approved of STL use but initiated nothing, so that teachers 

Integrating STL into their lessons did so with minimal support. 

Administrative leadership . Behaviors found to be Important influences on 
project implementation and institutionalization were specifically defined, and 
participants were asked to assign ratings on each behavior for central office 
staff and school-based administrators. (See Table 40.) 

Affective leadership behaviors were evident, with both school-based 
administrators and central office staff demonstrating commitment, advocating 
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* It is of Interest that energy remained relatively high in Worcester since 
that was an Independent site. In the summer of 1983, l^ received special 
recognition and was named as a demonstration site by the STL developer. 
Also, the principal and curriculum coordinator remained enthusiastic in 
support of STL teachers. 
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Table 40 



Administrative Leadership Behaviors: 
Student Team Learning, 1983-8A 



Behaviors 


Mean Rati 


ngs Assigned 




To Central 
Office Staff 
N-A3 


To oCnooi oasea 
Administrators 
N-48 


Affective 






Demonstrate commitment 


3.93 


4.00 


Provide support 


3.74 


3.82 


Logistical 






Press for fidelity 


2.16 


2.10 


Press for intensity 


2 .33 




Provide assistance 


'>.86 


3.83 


Coordinate LEA connmrnication 


3.59 


3i40 


Coordinate school communication 


3.37 


3.40 


Implement data-based 
decision-making 


3.30 


3.24 



Scales range from 1.00 (not at all) to 5.00 (to a very large extent). 
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the value of STL, and providing support by demonstrating Interest and 
enthusiasm and recognizing teachers' successes. 

Logistical leadership behaviors were evident to some extent for both role 
groups, with organizational process ratings higher than those related to 
"press." The relatively high ratings for assistance Indicated provision of 
information, materials, and training, and effective responses to requests for 
help. Since this administrative behavior (assistance) was strongly encouraged 
by MSDE (and It Influenced teachers' success), these ratings were positive. 
Communication ratings were good. Indicating that information exchange for STL 
was somewhat above what was common for other projects. Data-based decision- 
making for SITIP was also somewhat better than for any other local project; 
MSDE had hoped that this behavior would Increase. The low "press" ratings are 
important, and reinforce data from observations and interviews that, in 
general, teachers used STL at their discretion. While school administrators 
beli-ved that central office staff wanted STL to be implemented as intended by 
the developer (with fidelity) , and to be used regularly and often enough to be 
considered part of the instructional program (intensity) , all other role 
groups perceived press for fidelity and intensity to be slight to moderate. 
Teachers rated "press" from central office as very slight (1.91 for fidelity, 
and 1.93 for intensity), and from school-based administrators as slight (2.00 
and 2.15). They did not feel "held accountable" for using STL, but rather 
felt they had autonomy in deciding on STL use. These findings may not be 
considered particularly negative from the point of view of several LEAs since 
their planned purpose was to provide teachers with an additional instructional 
approach through staff development, with voluntary implementation. However, 
it should be understood that when fidmlnistrative and supervisory leadership 
does not Include strong clear expectations of fidelity and intensity of use, 
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application may gradually decline. This occurred in at least three LEAs, with 
adniniatrators stating in one case that they "couldn't even find any teachers 
using STL at. . .(schools) now." 

Decisions about classroom use of STL were primarily made by teachers; 
instructional leadership had a greater classroom focus for STL than for other 
models. In all sites, there was a heavy reliance on key teachers who had been 
trained by STL developers and MSDE TAs at MSDE follow-ups and institutes. 
Those key teachers trained others within their own schools, and (in three 
LEAs) trained teachers from other schools. In two cases, training beyond the 
initial school was conducted through a teacher center, with voluntary partici- 
pation and use. ia another case, an LEA training team was made up of key 
teachers from all six participating schools (to provide mutual support and 
exchange materials). Only at one of the teacher centers was training the 
primary responsibility of a central office staff person. This strong emphasis 
on teachers as instructional leaders was unique to STL, and was influenced by 
the perception of the model as an additional instructional tool, a classroom- 
centered activity to be used at the discretion of the teacher. 

Overall, interactive support for STL was good. Leadership was shared', 
with leadership staff providing encouragement and logistical assistance, and 
teachers undertaking primary responsibility for the nature and extent of 
implementation. Expansion occurred only when administrative and supervisory 
staff were involved and made provision for teacher release time for training 
and development of materials. Materials were most often developed in 
conjunction with initial training. After that, even strongly committed 
teachers found almost no time for materials development, and so used STL only 
for the particular unit initially addressed. 
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Outcomes 

Participants perceived that STL had an Impact on students, teachers, 
schools, and school systems. No LEA provided student assessment data to 
support perceptions. For Year 3 of the study, greatest attention was given to 
outcomes relating to Institutionalization. 

Students . While there was strong agreement across all role groups that 
students enjoyed STL (mean - 4.47), there was somewhat less certainty about 
improvement In achievement (mean - 3.69). (See Table 41.) There was some 
concern that poor students "got away with less effort." Intense use of STL as 
an appropriate assistance technique for students falling the Maryland 
Functional Mathematics Test was successful in one case where students moved 
from the 40th to 90th percentile. However, other educators found that overuse 
reduced STL's effectiveness. In generj^l, students looked forward to STL 
lessons, and welcomed the change from the "lecture" format. Elementary 
teachers were more certain of STL's Impact on students than were secondary 
teachers, but when the responses of all role groups were comoined, there was a 
consensus that the greatest benefit was the extent to which STL motivated 
students. 

Teachers . Teachers learned to use a new and effective teaching strategy 
that helped provide variety In their Instruction. STL was used for selected 
units of Instruction at the discretion of the teacher. Most teachers found it 
easy and inexpensive to Implement. However, some were concerned about the 
extra time required for planning and record keeping. 

In the STL classrooms observed, teachers were well-organized. Students 
understood the process for each STL strategy (i.e., STAD, Jigsaw, TGT) and the 
objectives to be accomplished (l.e.» student cooperation and competition). 
Some teachers used the materials developed by the STL developers at Johns 
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tab 1ft A I 



Instructional Inpact as Percalvsd by 
Survey Ratpondants: Studant Taaa Laaming, 1983-L984 





Role Groups 


lapact on Instruction 


CO 
N-5 


SA 
N-10 


T 
N-39 


Total 
N-54 


Inatructlonal Valua 










Works In classroom 

Is worth tha work It takas 


4.60 
4.20 


4.70 
4.50 


4.21 
3.79 


4.33 
3.96 


iBpact on Taachars 










Taachars enjoy It 

Taachars have Incraaaad knowladga 

Taachars have Increased skills 


4.20 
4.60 
4.60 


4.40 
4.33 
4.33 


3.89 
3.78 
3.54 


4.02 
3.96 
3.77 


Impact Of/ Students 










Students enjoy It 

Students* achievement has Increased 
Students are learning and retaining more 
Students' general behavior Is better 
Students are taking more responsibility 

for their own learning 


4.80 
3.40 
3.60 
3.80 

4.20 


4.67 
3.89 
4.00 
3.89 

4.33 


4.38 
3.68 
3.55 
3.50 

3.59 


4.47 
3.69 
3.63 
3.60 

3.77 



Mean ratings range from 1.00 (strongly disagree) to 5.00 (strongly agree) 
CO - Central Office; SA - School Administrators; T - Teachers 
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Hopkins University. However, the majority of teachers developed their own STL 
materials. Teacher enthusiasm for STL was influenced by the MSDE TA's 
commitment and the ongoing involvement of STL developers (e.g.. at STL 

follow-up sessions). 

Impact on students and teachers combined was defined as instnictlonal 
gain. There were no significant differences among the counties or between 
elementary and secondary schools on instructional gain. 

Schools . As indicated in Table 41. educators believed that STL worked in 
the classroom and was worth the effort. In some schools, cross-grade 
articulation improved as well as cross-hierarchical communication and 
cooperation. Of the 101 statements about program benefits. 32X related to 
logistics <e.g.. easy and inexpensive to use) and 31t to improvement in 
student behavior and/or performance. Of the 36 negative comments. 83Z related 
to the instructional program (e.g., time involved for planning, record-keeping) 

School organizational outcomes were fairly well established with the 
exception of regular participation and "ownership." (See Table 42.) Teachers 
had a lower perception of all five outcomes, especially "ownership" of SHIP. 
Ratings assigned indicate the relative "health" of STL. 

Policies for management and decision-making were also fairly well 
established. Teachers felt that data-based decision making occurred to a 
moderate extent compared to administrators who felt that it occurred to a 
fairly large extent. (See Table 43.) 

Procedures used to institutionalize STL were perceived to occur at a 
moderate to fairly large extent. (See Table 44.) Teachers perceived these 
procedural outcomes as occurring to a lesser extent than did administrators. 

Low mean ratings on institutionalization indicators, especially when ' 
accompanied by low ratings on leadership behaviors, suggested project decline. 
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Table 42 



Organizational Outcomes: Student Team Learning, 1983-84 



Oi tcomes 


School 


System 


kk- 


N 


Mean 


N 


Mean 


Cognltlve 

Status of SITIP established 


51 


3.78 


9 


4.11 


Close to 1003; of teachers asked 
tn nflr^^r>^nAt^ Ho bo resularlv 


48 


2.79 


9 


2.89 


Affective 










Local educators feel "ownership" 
of SITIP 


49 


3.34 


9 


3.44 


There is harmony between teachers 
and school-based administrators 
about SITIP 


51 


3.88 


7 


4.28 


There is harmony between school- 
bnsf-'d and central office staff 
about SITIP 


47 


3.81 


8 


4.37 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent). 
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Table A3 

Policy Outcomes: Student Team Learning, 1983-8A 



Policies 


School 


System 




N 


Mean 


N 


Mean 


Management Is shared 


lA 


A.IA 


9 


3.67 


Decisions are data based 


A9 


3.31 


8 


3.25 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent). 
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Procedural Outcomes: Student Team Learning, 1983-8A 



Procedures 


School 


System , 




N 


Mean 


N 


Mean 


Inservlce modified to support SITIP 


51 


3.9A 


9 


3:78 


Staff asslgments and accountabili- 
ties modified 


A9 


3.A1 


9 


3.22 


Resources allocated annually 


A9 


3.51 


9 


3.67 


Local funds used 


lA 


3.57 


7 


3.71 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent). 
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Such results were apparent In one LEA. However, overall data suggest that in 
LBAs responding to the survey, STL is being institutionalized at the school 
level although scope and intensity of use is relatively low. Other data 
suggest that as project status ends, institutionalization slips to the 
classroom level and STL use is determined by individual teachers. 

School aystem . In the two LEAs where leadership and constructive 
Involvement of the central office was intended, system level outcomes were 
particularly important. They were less important in the other systems where 
schools took greater responsibility for STL projects. 

Of 11 positive comments about system outcomes, 46% related to logistics 
and 36% to process. No negative comments were made. 

Organizational outcomes at the system level were apparent to a fairly 
large extent across the LEAs with the exception of regular participation and 
"ownership" which were established to a moderate extent. System level 
organizational outcomes were more firmly established than school level 
outcomes. (See Table 42.) 

Policy outcomes were moderately to fairly well established. (See Table 
43.) Teachers were involved in the management of the project, and decisions 
were made based mainly on student and staff enjoyment of STL. Procedural 
outcomes were also good with inservice receiving the highest mean rating. 
(See Table 44.) * ' 

System-wide institutionalization was strongly indicated for two LEAs, one 
of which had been involved for three years and one for two years. One used a 
pilot district strategy concentrating on the elementary level, and one used a 
capacity building approach at the secondary level. 

Four LEAs did not respond to the survey question. One LEA, using a pilot 
district strategy began the project in one middle school, and because of Its 



" success, the two feeder elementary schools joined the project during Year 3. 
STL appeared to be well established In these schools. The second LEA 
l?»plemented STL In one junior high school with no Intentions of system-wide 
Instltutlonalliatlon. The remaining two LEAs Implemented STL Independently, 
with teachers voluntarily using STL at their own discretion. 

Relationships among outcomes . Data were grouped Into the following 
indices: Instructional gain, system Institutionalization, school 
institutionalization, central office support, and school administrator 
support. Correlational analyses were conducted to determine the relationships 

amoAg these indices 

Correlations among the five indices showed strong direct relationships 
(r ^ .50) betvaen syatem institutionalization and two othei^ ilwUcfifl — 
central office support and school institutionalization. School institution- 
alization was strongly correlated with school administrator and central office 
support. Central office support and school administrator support were 
strongly correlated. (See Table 45.) 

Influences and plans . Influences on past implementation and future 
action included cnvlronmentaal turbulence such as: (1) staff changes 
resulting in shifts in SITIP leadership. (2) program changes resulting in 
revising priorities or restructuring tasks, and/or (3) organizational changes. 
They also Included the nature and extent of outside (MSDE) assistance, and 
local perceptions of program success. 

All LEAs except one experienced some type of environmental turbulence, 
leadings — in five cases — to a reduction in the number of teachers involved 
and/or the frequency of use of STL. Four of the five declining projects were 
Independent, and while two of them maintained involvement with STL 
developers, that Involvement did not appear to compensate for changes in local 
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Table A 5 



Intercorrelatlons Among the Indices; Student Team Learning 



Index 


1 


2 


3 


A 


5 


I. 


Instructional Gain 




N.S. 




• 


ill 


2. 


System Institutionalization 






.72 


.99 


N.S. 


3. 


School Institutionalization 






.5A 


.85 


A. 


Central Office Support 








,66 


5. 


School Administrator Support 











N.S. - Not significant at the .05 level. 

Note ; The number of cases upon which the correlations were 
calculated varied. 
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administrators' commitment. Perceptions of project success stimulated the 

effort to overcome the negative influence of environmental turbulence in some 

oases. However, there is an indication that such perceptions may not be 

> 

strong enough to ensure future widespread institutionalization. 

By the end of Year 3» for the LEAs formally involved in SITIP, the two 
LEAs with system-wide approaches had involved 29% of their 2A schools. For 
Year 4» one LEA planned to maintain current levels of implementation and the 
other hoped to expand the model to other subjects and fclasses within the 
implementing schools. The remaining three LEAs (with school-based approaches) 
Involved 10% of their 187 schools by June 1984. All three focused on mainte- 
nance with some expansion into other subjects and/or schools. The four- 
independent LEAs planned to allow voluntary usage of STL with training 
opportunities for Interested educators provided by two LEAs. 

Specific activities and factors influencing plans are presented in Table 
46. In general, most participants (64. 8Z) planned to maintain their current 
level of implementation, primarily because students arid staff liked STL. 
Expansion was planned mainly in classes within implementing schools rather 
than in additional schoo^^s or through additional models, ^nservice training 
and resources will be provided by some LEAs. Plans f^'^reduction were minimal 
and were Influenced by perceptions' that STL was not cost-effective. 
Summary and Conclusions 

Over the three years of implementation, LEA involvement decreased from 
nine to five counties formally participating in SITIP. Of the nine LEA,8, by 
the end of Year 3 teachers in four counties (in from one to 10 schools per 
LEA) were using STL on a voluntary basis without MSDE funding or formal 
support from central office staff. Of the remaining five counties, two 
changed their implementation strategies during Year 3, all increased the 
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Table 46 



Activities Planned and Influences Perceived 
by Student Team Learning Implementers (N-54) 



Type of Activities 


X Respondent u** 




X Respondents Pe 


r Factor* 






1 


2 


3 


4 


5 


6 


7 


Maintenance 

*Xlo4n^fl4n nirrAnt! level 


64.8 


18.4 


10.2 


24.5 


6.1 


1.9 






Allow voluntary use 


35.2 


9.8 


1.9 


17.6 


1.9 








Expansion 

l!iXpanQ CXooocSy 


42.6 


8.2 


6.1 


14.3 


6.1 








Exoand - schools 


9.3 


1.9 


1.9 


3.8 




1.9 






Add another component 
of model 


9.3 


3.8 


3.8 






1.9 






Add a new model 


5.6 


1.9 




3.8 










Support 
Conduct inaervice 


13.0 


1.9 


1.9 




3.8 




1.9 




Provide resources 


22.2 


3.8 


3.8 


7.7 


1.9 




1.9 




Reduction 

Discontinue SHIP 


1.9 














1.9 


Reduce activity by 
25% or more 


5.6 














i 


Totals 




49.7 


29.7 


7.1.7 


19.8 


5.7 


3.8 


1.9 



* Ten factors were listed on the survey, of which seven wer^ indicated as 
influential in planning by some respondents. 



1. High student achievement data Indicate SITIP value. 

2. SITIP helps achieve a local priority. 

3. Students and staff like SJTIP. 

A. Teachers' instructional skills improve with SITIP. 

5. Senior administrator's advocate SITIP. 

6. Funding cuts: ojther things take priority over SITIP. 

7. SITIP is not cost-effective 

8. There is little staff support for SITIP. 

9. Senior administrators have little interest in SITIP. 
10. SITIP has very little td do with local priorities. 

** Many respondents checked more than one type of plan. 
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number of students Involved, and three Increased the numbers of teachers and 
schools Involved. 

STL was used in a wide variety of subject areas In, both elementary and- - 
secondary grades. Fidelity was fairly high ani concepts of the model well 
understood. The instruction provided In the STL classes observed was good, 
with teachers finding STL an effective alternative instructional process. STL 
was most often used periodically at the discretion of the teacher for 
reviewing certain concepts within a subject area. STL was not used 
consistently for an entire course. On average, teachers used STL for about 

IQX of their in-class time. 

STL was perceived by most of the educators involved as a teacher-centered 
strategy. Teache.-s took primary responsibility for implementation. They 
planned lessons, developed materials, and trained other teachers. Central 
office staff and school administrators, in most cases, provided affective and 
logistical support, but did not "press" for fidelity and intensity of 
implementation. Low "press" contributed to the gradual decline in model usage 

experienced by some LEAs. 

Two of the SITIP LEAs (one using school-based and one a system-side 
approach) had high ratings of institutionalization at both the school and 
system levels. Implementers in both LEAs received support from both central 
office staff and school administrators. Local plans Included maintenance of 
current levels of implementation and expansion to other schools in the 
district. 

Two SITIP LEAs (one using a system-wide and one a school-based approach) 
had high ratings of institutionalization at the school level. Implementers In 
both LEAs received more support from school administrators than from central 
office staff. 
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One SITIP LEA appeared to be in a state of gradual decline that may lead 
to termination. This LEA suffered environmental turbulence in the form of 
staff reassignments which led to a change in project leadership and Increased 
workloads for impleraenters . During the first two years of the project, 
central office interest and commitment was minimal.* In Year 3, principal 
support '-t the lighthouse site was aiso low. Due to this lack of interest by 
administrators, teachers decided to spend their limited time on other priority 
areas. 

In general, local educators experienced success in implementing STL. 
They believed that students' attitudes toward learning were more positive, and 
teachers acquired a new instructional strategy. Belief that students and 
teachers enjoy STL was stronger than belief or evidence that STL helped 
increase student achievement, and It was the former that influenced plans for 
Year 4. Most LEAs plan to continue using STL on a voluntary basis at the 
Leachers' discretion. Some LEAs plan to expand STL to other classes and 
schools. Others plan to provide training for interested educators. It is 
hoped that LEAs will make appropriate project decisions based on the 
recognition that (l) MSDE support will gradually be reduced, (2) LEAs with 
indications of project decline might choose to terminate, (3) school-focused 
sites might independently institutionalize at that level, and (4) district-wide 
use might continue to need extra effort from administrators and supervisors. 

Iraplementers from the four independent LEAs will probably continue to use 
STL on an individual voluntary basis with minimal central office or school 
administrator support. Two LEAs have provided training opportunities for 
interested teachers and have ties with the model's developers which may help 
keep STL usage alive. 



* In Year 3, central office staff attempted to improve rapport and support. 



LEAfl planning to continue using STL may wish to review the Indicators and 
findings discussed In this report and monitor their progress accordingly, 
applying techniques and strategies to make Improvements.* 

Of general concern are: (1) the emphasis on voluntary use by teachers, 
(2) the fairly widespread belief that STL Is more appropriate for occasional 
motivation than for a complete unit of Instruction, (3) the low press for 
fidelity, and (4) the relatively moderate Impact of STL on achievement. If 
teachers have appropriate materials they use STL, but if fewer than three 
teachers in a school are involved routinely, vie declines. If STL Is used 
properly for a complete unit of instruction, student achievement increases, 
but low-fldellty STL used sporadically may make no difference. Administrative 
press for fidelity and intensity encourages appropriate use, which should 
result In increased student achievement. Plans and strategies through June 
1984 were most influenced by students' and teachers' enjoyment, but If they 
were equally Influenced by increased achievement more widespread 
Institutionalization of STL might well occur. 

Teaching Variables (TV) 

This section describes the local implementation of Teaching Variables 
(TV). The Information Is drawn frow various sources, including survey 
questionnaires completed by 74 local Implementcrs (eight central office staff, 
15 school administrators, and 51 teachers). All TV LEAs were repreawnted. 
Approximately 8U of the responding teachers were In elementary schools. 
Olocusslon focuses on: planning; the scope and Intensity of use; the roles 
and responsibilities of Isiplemontcra; the outcomes in terms of Impact on 
students, teachers, schools, and school ayotema; and Influences and plans for 
Year 4. The section surawary reviews the three years ot' Implementation and 
makes wiorae recommtnidtttions for the future. 
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Planning 

As stated earlier, TV as developed is a complex model with activities 
related to two variables: "time" end "content." Attention to "time" requires 
systematic data collection through classroom observation and strategizirg to 
improve student time on task. Attention to " ontent" requires curriculum 
alignment and diagnostic/prescriptive instruction. As dfesigned, the model 
relies primarily on knowledge based on elementary schools. While all six TV 
LEAs adopted the "time" variable, only three addressed "content" to some 
extent, thus reducing complexity. TV was most often used in basic skills 
classes, but was also used for other subjects in four secondary sites. LEAs 
perceived the model primarily as a method to collect data and to analyze use 
of time and content coverage. 

An analysis of local plans for the 1983-84 school year identified LEA 
objectives at the beginning of the year (September 1983), and the extent to 
which project coordinators considered those objectives to be achieved at that 
time ,md also at the end of the year (June 1984). (See Table 47.) In each 
case, the percent of LEAs that "hoped for," "partly achieved." or "achieved" 
each objective was indicated. There were nine objectives Identified, all 
partly achieved by at least 33% of the LEAs at the beginning of the year 
(having been addressed in Years 1 and 2), and some maintaining that status in 
June. Progress was made in all areas, and was strongest in curriculum 
alignment and helping teachers become better organized. Of the six LEAs 
Involved, one (Kent) made progress toward systemic achievement of objectives. 
In that county, and in Calvert and Montgomery, informing and training other 
educators were objectives strongly addressed. 

S cope. Intensity f and Fidelity of Use 

Scope indicates the number of schools, teachers, students, and so forth 
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Table 47 

Status of Local Objectives, 1983-84: Teaching Variables 







Percent of Sites Achieving Local Objectives 




Local Objectives 




Pre- (Sept. 


1982) 




Post 


-(June 


1983) 






Status of Achievement* 




Statufe of Achievement* 






N 


I 


2 


3 


M 


1 


2 


3 


1. 


Improve student achievement 
(basic skills) 


6 


0 


67 


33 


6 


0 


50 


50 


2. 


Improve student achievement 
)ther subjects) 


6 


33 


50 


17 


6 


17 


50 


33 


3. 


Inform local educators about model 


5 


0 


80 


20 


6 


17 


33 


50 


4. 


Train educators to use model 


6 


0 


67 


33 


6 


29 


42 


29 


5. 


Improve teachers* classroom 
competence 


6 


17 


67 


17 


6 


0 


67 


33 


6. 


Ensure match of Instruction, 
curriculum, -«nd test(s) 




20 


40 


40 


5 


0 


20 


80 


7. 


Help teacherd become better 
organized 


6 


17 


33 


50 


6 


0 


57 


43 


8. 


Improve tlme-on-task 


6 


0 


71 


29 


6 


0 


57 


43 


9. 


Improve students' Involvement In 
learning (motivation) 


6 


0 


100 


0 


6 


17 


50 


33 



16'.) 



* ] « Hoped for 

2 ■ Partly achieved 

3 " Achieved 



Note : Total number of LEAs Implementing Teaching 

Variables equals 6. N equals the number of LEAs a ^ 
addressing each objective. 1 • U 



Involved. Intensity relate* to the amount of time a given teacher or class 
used TV. Fidelity is the extent to which teachers implemented the components 
of the model as Intended by the developer. Scope, Intensity, and fidelity in 
the classroom were influenced by administrators' investment of time, and the 
kinds of activities they carried out. Also influenced by the implementation 
strategy employed, these dimensions Indicate the nature and extent of use. 
Table 48 summarizes the use of TV for Year 3. 

Scope . Six LEAs implemented TV. Five began as lighthouse sit3s, of 
which three changed. One began as a pilot/district, but switched to a light- 
house approach in Year 3. In practice for Year 3, there was one district-wide 
LEA (Kent, which switched from a lighthouse approach in Year 2), one pilot/ 
district (Calvert), three lighthouse sites, and one that began expanding from 
the lighthouse schools by using a capacity building strategy to train staff of 
several other schools (Montgomery) . In the elementary schools in Somerset and 
Montgomery, TV was combined with Active Teaching, with the latter model used 
as an instructional improvement strategy. By the end of June 198A, 17 schools 
were using TV as part of SITIP, of which six were elementary, nine were middle 
schools, one was a high school, and one a vocational-technical school. (In 
Calvert and Montgomery, teachers in other schools were trained , and In 
Somerset other principals were trained. Some of those trainees also used TV, 
but did so less systematically than at the SITIP sites. These "voluntary" 
users did not provide data for this study.) About 123 teachers were involved 
(25 more than at the end of Year 2). Increased involvement occurred in Kent, 
Somerset, and Talbot. Across the state, TV was used in al out '>% of the SITIP 
schools by about 5% of the teachers, and was the least ust d of the SHIP 
models. 

Intensity . All but one LEA began Implemetitation in ear i, and vaany 



155 



171 



T«bl« 48 

8cop« of IaplMMat*tloa. S«pt«ib«r 1983 tod Jun* 198A: TMchlnf VtrltblM 



LEA 


Strategy 


DlMOilont 


f of Schoolt 




f of TMchort 1 


f of Studwti 1 Subioet Ar— f 


Calvtrt 


LS/PD 


Sopt 83 


Juno 84 


SoDt 83 


Juaa 84 


8«pt 83 1 




ETIMfl EHM Email ESmil 


I 


3 


J/M 


J/M 


18 


18 


475 


468* 


R/U.N. 
Sc.SSiO 


R/U.N. 
be. 88.0 


Frftdtrlck 


PD/LS 


2 


2 


J/N 


J/M 


17 


14 


400 


350 


t/U.N. 

Sc. 88 


t/U.N. 
8a. 8S 


Kmt 


W 


7 


7 


l.J/H 


I.J/M 


36 


52 


1.430 


1.561 


B/U 


fl\A 


lfont|OMt7'»' 


L8/CB 


2 


2 


I.J/M 


B.J/M 


16 


14 


316 


400 


t/U.N. 


t/U.N. 
8e.88 


SoMrMt^ 


L8 


2 


2 


B.B 


B.B 


11 


12 


388 


420 


B/U.SS 


B/U,N, 

88 


Talbot 


IS 


1 


1 


0 


0 


12 


13 


253 


230 


0 


0 



* At pilot 
Strtf gy : 



school ~ ^3 of •tttdmitt %t othmt aiddlt tchoolt. 

•t one achool Mrt uaint T^tchlag Variabltt • dttt colltctlon ttchnlqut for Actlvt TMchlos. 



LS«Llghthouat achool 
PD«Pllot dlatrlet 
i:B>K:Apaclty bulldins 
DV«Diatrlct-«ldt 



Typo ; l«Bl«Mntory ochool 
J/M-Juaior hish/«lddlo 
H«Biih ocbool 
0-Othor 



SttbJoct Arf t R/lA-RMding/lofisuaio ortt 
M*IUth«Mtlct 
Sc^Scltftco 
88«SocUl ttudlaa 
(M)thor 
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teachers and administrators maintained involvement over the three years. The 

4 

average for administrative and supervisory -staff was just over two years. 
Teacherb, on average, wore involved for Just under two years, with longest 
participation in Montgomery and shortest in Kent. Mean times were influenced 
in part some late starts in Year 1, and in part by expansion site teachers. 
During the 1983-84 school year, teachers reported being involved 7.38 months 
on average, with the lowest mean one month and the highest the full year. 
Teachers reported involvement based on "time" observations. For instance, in 
one elementary school, teachers reported a full school year of involvement 
because each was observed by the principal at the beginning and end of the 
school year. However, since "content" was not addressed and little 
strategizing took place (since student engagement rates were high). Intensity 
of TV use was minimal. At two other sites where teachers conducted 
observations of each other, in one case mean involvement was reported as 4.25 
months and in another as 7.50 months, yet the intensity and frequency of 
activity were very similar at both sites. For these reasons, the self-report 
data do not present a reliable picture of implementation intensity. Other 
data sources indicate the following. 

• In all but one site participating teachers were observed for "time-on- 
task" at least tv;ice during the school year, and in some cases as many 
as seven times. An "observation" at one LEA took one full class 
period. At another, it was made up of three separate 10-minute "mini" 
observations, and at others was a single visit for part of a lesson. 

• Teacbers involved in "content" usually addressed one subject, in 
elementary schools for the full year, and in secondary schools for a 
given course and grade level. 

Fidelity . TV has two variables. Key "time" components include: (1) 

systematic collection of time-on-task data to determine student engagement 

rate; (2) discussion of that data, either in a group or one-on-one between 

observer and the teacher observed, to determine strategies for Improvement; 
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(3) application of Improvement strategies (If needed); and (4) repeated data 
collection to check on Improvements. Observations may be conducted by 
admlnlatratora, supervisors or by other teachers. "Content" may be considered 
as a kind of lesson planning/review process, with the following key 
components: (1) analysis and matching of objectives, curriculum, and 
assessment (using such standardized tests as the CAT) ; (2) knowledge of 
students' prior learning of related/ prerequisite skills; and (3) systematic 
record-keeping of content coverage (by objective) and student achievement. Of 
the 46 teachers responding to the general survey 78% addressed "time'' and 52% 
addressed "content". Tlme-on-task data were collected for 74% of the 
teachers: 59% by peer observation, 57% by principals, and 35% by central 
office staff. (Some teachers were observed by more than one role group.) 
Discussion and strateglzlng was, for 67% of the teachers, one-on-one. For 
57%, no Improvements were considered necessary since engagement rate was high. 
For the remainder, some changes In classroom management were made and 
tlme-on-task Improved. Where no changes were made, and only the "time" 
variable was addressed. Implementation consisted virtually of data collection 
and analysis — hlg "idellty, low Intensity, with Instructional Improvement 
very unlikely. For "content," 46% addressed curriculum alignment, with 16% 
modifying curriculum or Instruction following analysis, 26% keeping records, 
and* 33% being aware of students' prior learning. Since curriculum alignment 
Is done statewide for Project Basic, much ground work had been completed. 
However, teachers could also have applied lesson-specif Ic outlines suggested 
In TV guidelines; few did so. In two LEAs teachers were totally unlnvolved In 
"corttent;" In three LEAs, "content" activities were not systematic, and In two 
others activities were fairly systematic, with one (Calvert) using 
teacher-developed checklists for several subject areas and grade levels. 



Fidelity was Influenced by administrative "press" and by participants' 
understanding of the model. "Press" for fidelity and intensity was fairly 
low, and understanding varied from site to site. Although understanding 
(particularly for "content") was very good across all role groups in at least 
two sites, was mediocre in others, and in one LEA was very poor (resulting in 
extremely low fidelity). Relative understanding was influenced by training: 
emphasis on coding distorted the model toward analysis and away from 
strategizing and implementing for instructional improvement. 

Administrative investment . The average amount of time invested in SITIP 
by an administrator was 20.50 days during Year 3, with central office staff 
spending less (mean - 18.33 days) than school-based administrators (21.43 
days). Activity areas, in order of priority allocation of time, included: 
(1) monitoring/evaluation, (2) administration/communication, (3) general 
support, (4) inservice. and (5) dissemination/expansion.* Where administrative 
and supervisory staff conducted time observations, monitoring was identified 
as the highest area of activity except in one LEA where principals 
(responsible for observation but not perceiving that as monitoring) spent most 
of their time on inservice. In the remaining two LEAs, where teachers carried 
out observations and also conducted training, administrators and supervisors 
spent most of their time on communication/administration. 

In general, the quality of implementation was patchy. The greatest weak- 
ness of TV implementation was the lack of strategizing for improvement. The 
greatest strength was evident in sites where understanding of the model was 
high and teachers strateglzed with each other. The emphasis on "time" coding 
was such that post-observation "discussion and strategizing" was (in some 
cases) simply a statement of the engagement rate and the fact that it was good 
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* See Roles and Responsibilities for a more detailed discussion of activities. 
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enough not to warrant Improvement activities* At It's worst, data were 
collected » teachers got feedback only if they asked, and the project was 
perceived not as a means of instructional improvement but as data collection 
without practical implication* At its best, TV provided a data-base for staff 
decision-making about instructional improvement, particularly in relation to 
difficult teacher-*student relationships or classroom management, or It 
provided a focus for professional development by and for teachers.' 

Roles and Responsibilities 

The SITIP design encouraged involvement of cross-hierarchical teams. By 
the end of Year 2, it was apparent that: (1) teachers involved In MSDE 
training activities sometimes became instructional leaders, or "key" teachers, 
who provided training or assistance to others In their LEA; (2) school-based 
administrators trained by MSDE and developers were more commit ed to 
Implementation than those trained by "turnkey" trainers; but (3) central 
office staff (with only a very few exceptions) did not appear to have gained 
much from MSDE-sponsored training, since in all but one LEA they were not 
directly Involved In Implementation, and sometimes had little understanding 
of the model. In general for TV, central office staff undertook 
administrative tasks, principals were supportive, and teachers moderately 
active* In two sites, leadership tasks were undertaken primarily by teachers, 
but otherwise administrative and supervisory staff made decisions and 
organized activities. Project success was Influenced by local "messages"^ of 
Implementation and environmental turbulence. 



* "Messages" of Implementation Included overt statements of enthuMasm, 
acknowledgement of teachers* successes and administrative «nd supervisory 
staff efforts, as well as such more subtle notions as sincerity In 
considering points of view, building shared understanding and trust about 
project purpose, and valuing the model. 

17 V, 
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In Year 3, project staff needed to correct errors, consolidated successes, 
add the "content" variable if so jilanned, and make data-based decisions about 
use of the model as state funds were withdrawn. Local staff were encouraged 
to apply the recommendations made in earlier SITIP reports, with particular 
attention to central office staff involvement. The discussion below describes' 
^ role group activities more specifically in terras of interactive support and 

leadership behaviors* 

Interactive support . Support, among LEA participants included exchanging 
Information and materials, providing formal training, coaching and trouble- 
shooting, managing logistical arrangements, and recognizing successes. 
Support from MSDE and TV developers consisted primarily of training, with some 
technical assristance and trouble shootings In Year 3, developers had no inter- 
action with LEAs although in previous years they conducted on-site training in 
three LEAs and participated in follcH^-up training conducted by MSDE, They also 
provided some assistance to one secondary school site. Survey respondents 
rated support received (on a five ppint scale from l,0O"very poor to 5.00« 
excellent), and meait ratings assigned by each role group are presented in 
Table A9. Overall ratings ranged from a low of 2,26 (for developers) to a 
high of 3,78 (for teachers). Among LEA role groups, teachers received average 
to good ratings from all, and (as in previous years) assigned lower ratings 
than did other role groups. 

Ratings within each county indicated the fpllowiftg, 

• In all LEAs, teachers were awarded above average ratings, but iu only 
two counties were they rated higher than other role groups, and in one 
LEA they received lower ratings than administrativ'e and supervisory 
staff. 

• With the exception of one LEA school administrators were awarded above 
average ratings, and in three counties they were rated higher than 
other role groups, 

• Average or above ratings were awarded to central office staff In four 
LEAs, but in two cases their ratings were lower than for the other twr 
local role groups, 
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Table 49 

Perceptions of Support Received: Teaching Variables. 1983-84 

Support Groups 



Respondents 



Central Office Staff 
School Administrators 
reachers 

Total 



Teachers 



N 



7 

14 
51 

72 



Means 



3.86 
4.21 
3.65 
3.78 



School 
Administrators 



N 



7 

10 
51 
68 



Means 



3.14 
4.30 
3.73 
3.75 



Central 
Office Staff 



N 



7 

15 
51 
73 



Means 



3.14 
3.47 
2.73 
2.92 



MSDE 



N 



8 
15 
51 

.74 



Means 



3.50 
3.67 
3.02 
3.20 



Mean ratings range from a low of 1.00 (poor) to a high of 5.00 (excellent). 



Developers 



N 



Means 



5 

10 
47 
62 



2.60 
2.90 
2.09 

2.26 
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• Low ratings Indicated low interaction and respondents' perceptions of 
relative disinterest on the part of a given role group. 

In comparison to previous years, mean ratings across LEAs were about the 
same for all role groups, with vei> -UKht increases for teachers and school 
administrators, and decreases for the other role groups. Since expansion was 
minimal, it was not surpriaing that there were few changes. 

Administrative leadership . Behaviors found to be important influences on 
project implementation and institutionalization were specifically defined, and 
participants asked to assign ratings on each behavior for central office staff 
and school-based administrators. (See Table 50.) 

Affective leadership behaviors were evident, with administrative and 
supervisory staff demonstrating commitment, interest, enthusiasm, and 
recognizing teachers' success. Ratings were slightly higher for school-based 
administrators. 

Logistical leadership behaviors were evident to some extent for both role 
groups with organizational process ratings higher than those related to 
"press," but .lone much above a moderate level except for provision of 
assistance from school administrators. The latter indicated provision of 
information, materials and training, and effective responses to requests for 
help. Since this administrative behavior (assistance) was strongly encouraged 
by MSDE (and it influenced teachers' investment of effort) this rating is 
positive. School communication ratings are good, but mean central office 
scores reflect the low participation of that role group in most sites. Data- 
based decision-making was a little better at the school level for TV than for 
other programs; MSDE had hoped this behavior would increase. 

In three LEAs, for six secondary schools, leadership was undertaken by 
teachers (with strong administrative and supervisory support). In each of 
,t,-~- o/«KnAic (^wo T.FAo^ key teacher teams ran independent projects. 
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Table 50 



Administrative Leadership Behaviors: 
Teaching Variables. 1983-84 





Mean Ratings Assigned 


Behaviors 


To Central 
Office Staff 
(N-53) 


To School Based 
Administrators 
(N-60) 


Aiiective 






Demonstrate commitment 


3.35 


3.95 


Provide support 


3.11 


3.80 


Logistical 






Press for fidelity 


2.04 


2.23 


Press for Intensity 


2.04 


2.32 


Provide assistance 


3.00 


3.62 


Cootdlnate LEA communication 


2.69 


3.00 


Coordinate school communication 


2.68 


3.30 


Implement data*based 
decision-making 


2.88 


3.17 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent). 
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In the third lEA, a key teacher team at one school trained other teams, and 
maintained communication among the schools. In that LEA, administrators were 
fully informed of proje-t successes and problems (and facilitated expansion), 
but in the two oth^r districts administrators were relatively uninformed. In 
all three LEAs. actual data (collected by peer observation) on specific 
classes were not used in teacher evaluation. Independent teacher 
implementation was relatively unsuccessful in one LEA where turnkey training 
and initial administrative purposes resulted in misunderstanding of the model, 
so that teachers implemented it Incompletely. Problems went unrecognized Irt 
part because the project was teacher-led, and newly assigned administrative 
and supervisory staff assumed complete, high-fidelity implementation was 
occurring. In the other two LEAs, where key teachers had been Initially 
trained by developers, fidelity was good.* 

Overall, interactive support and leadership for TV was fairly good. 
However, in four LEAs central office staff involvement or understanding was 
low, and in two cases where staff reassignments were made the new coordinators 
did not resolve problems of fidelity and harmony by the end of the year. Where 
central office support and involvement were high, fidelity and scope were high 
and harmony among role groups Improved in Year 3. School administrators were 
involved and well-informed in all but three schools (where principals were 
newly assigned). While supportive of teachers' efforts in those schools, the 
principals did not perceive that TV could help them achieve their priorities, 
and their lack of understanding contributed to very poor Implementation in two 
cases. Teachers worked hard as project leaders and observers. In general 



* Administrative staff of both LEAs were also trained with teachers. In one 
case, continuity was maintained. In the other case, administrative staff 
w( re reassigned. Commitment, scope, intensity, and understanding of the 
model were better at the former site. 
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Teachers appeared to be more passive than active since only 52% made changes 
related to "time," and 35% made changes related. to "content," 

Outcomes 

Participants perceived that TV had an Impact on students, teachers, 
schools, and school systems. No LEA provided student assessment data to 
support their perceptions. For Year 3 of the study, greatest attention was 
given to outcomes relating to institutionalization. 

Students. All three role groups assigned ratings between "not sure" 
(3.00) and "agree" (4.00) in relation to TV's Impact on students. (Sae Table 
51.) The highest mean score (3.58) was given for Improvement of students' 
behavior. Students understood the "time-on-task" components, and 24% of the 
survey respondents said that students attended to improving their engagement 
rates; 12.5% reported improved grades, and 7% said that students' attitude 
toward learning Improved. Engagement rates reported were usually high (above 
85%), and ranged from 69.3% (average from four observations of a middle 
school social studies class), to 91.6% (average of seventeen observations of 
basic skills in a vocational-technical school). No school system reported 
achievement data for participating classes. 

Teachers . Teachcrfc^' knowledge of effective teaching and skill in 
instruction Improved. Peer observation contributed to the professional growth 
of observing teachers, was considered non-threatening, and facilitated teacher 
cooperation at one site. However, at another site the observers complained 
that TV logistics (planning for substitute teachers, scheduling, processing 
data) took too much time, and teachers observed complained that results were 
not readily available. Where administrators conducted observations » thp focus 
on students and attention to classroom management were valued, but there ware 
some concerns that the coding was no better than "regular" classroom 
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Table 51 



Instructional Impact as Perceived by 
Survey Respondents: Teaching Variables, 1983-1984 





Role Groups 


Impact on Instruction 


CO 

N«7 

11 » 


SA 
N»15 


T 

■ N-50 


Total 
N-72 


Instructional Value 










Works In classroom 

Is worth the work It takes 


4.43 
4.14 


4.40 
4.40 


4.02 
3.60 


4.14 
3.82 


Impact on Teachers 










Teachers enjoy It 

Teachers have Increased knowledge 

Teachers have Increased skills 


3.36 
3.86 
4.29 


3.60 
4.33 
4.27 


3.56 
3.84 
3.76 


3 .60 
3.94 
3.92 


Impact on Students 










Students enjoy it 

Students* achievement has increased 
Students are learning and retaining more 
Students' general behavior is better 
Students are taking more responsibility 
for their own learning 


3.33 
3.57 
3.29 
3.57 

3.14 


3.47 
3.73 
3.60 
3.73 

3.27 


3.48 
3.38 
3.32 
3.54 

3.35 


3.46 
3.47 
3.38 
3-. 58 

3.31 



Mean ratings range from 1.00 (sr.rongly disaijree) to 5.00 (strongly agree). 
CO « Central Office; SA - School Administrators; T - Teachers 
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observation and that principal support in some schools was low. Impact on 
teachers was discussed by participants in terms of "time" observations, with 
two expections where teachers worked together to strategize for improvement 
and address "content." In all other sites, strategizing appeared to be 
minimal and was not voluntarily discussed by participants. 

Impact on students and teachers combined was defined as instructional 
gain. An analysis of variance showed significant differences among the four 
SITIP models on instructional sain. The mean for TV on instructional gain 
(3.67) was .26 below the group mean for all models combined. There were no 
significant differences among the TV LEAs and between elementary or secondary 
schools on instructional gain. 

Schools . As indicated in Table 51, educators agreed that TV was worth 
the work it took. They found that in some schools increased awareness of 
"time" contributed to an orderly environment, and attention to "content" 
facilitated coverage of core objectives. Of 59 statei^ents about program 
benefits, 25% related to logistics, 25% to organizational process, and 21% to 
the overall instructional program, indicating that TV "fits in" with school 
practices and contributed to better use of classroom time.^ 

Of 45 negative comments, 40% related to organizational process, 
indicating a need for improvement of MSDE follow-up workshops and assistance, 
a need to reduce the frustrations perceived by some teachers, and a need to 
address the disinterest of some teachers and administrators. The 40% negative 
comr related to instr ction indicated that participants considered 
tlme-on-task not to be a problem, or that use of TV had no benefits. These 
concerns related to outcomes in organization, policy, and process ~ 
Indicators of institutionalization. 

School organizational outcomes were moderate, with teachers' perceptions 
lower than ratings by central office staff and school-based administrators on 
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all areas. Ratings assigned indicate the "health" of TV. (See Table 52.) 
Project status, participation, and ownership were only slightly established at 
the LEA where administrative reasslgnments occurred most and purpose was 
changed In Year 2. Teacher participation was also low in two other LEAs where 
purpose was clear and staff stable but initial resistance had not been 
overcome. "Ownership" was also low in the LEA using the district- wide 
appro^'c^i. 

Ratings on shared management policies reflect teacher leadership in three 
LEAs. Decision-making policies were fairly well established, although 
teachers believed that decisions were data-based only to a moderate extent. 
(See Table 53.) Very low ratings were assigned on the latter by teachers in 
the LEA where the project purpose changed. 

Procedures used to facilitate institutionalization of TV were moderately 
established. (See Table 5A.) Inservice and staff assignments were only very 
slightly modified in the LEA where purpose changed. 

Low mean ratings on institutionalization indicators, especially when 
accompanied by low ratings on leadership behaviors, suggested project decline. 
Such a result was clearly apparent for one LEA. In one other county, it is 
possible that TV use in specific schools will decline, although the projtjct 
may survive in the pilot school. 

School system . In the LEA with district-wide implementation, system 
level outcomea were particularly important. They were less Important where 
leadership was school-based. 

Institutionalization indicator ratings were all lower than those assigned 
to schools, with the exception of annual allocation of resources.'' 



* Teachers did not rate system factors, therefore scores are not depresised by 
that role group which consistently assigned lower ratings than did 
administrative and supervisory staff. 
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Table 52 



Organizational Outcomes: Teaching Variables, 1983-84 



<*• 

W LA ^ w uic o 


School 


1 

System 




N 


Mean 


N 


Mean 


Cognitive 










Status of SITIP established 


71 


3.59 


10 


3.10 


Close to lOOZ of teachers asked 
to parcicipace do so regularly 


71 


3.58 


9 


2.22 


Affective 










Local educators feel "ownership'' 
or aiiii' 


67 


3.06 


10 


1.90 


There is harmony between teachers 
and school-based administrators 
about SITIP 


71 


3.80 


7 


2.71 


There is harmony between school- 
based and central office staff 
about SITIP 


67 


3.24 


10 


2.90 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent). 
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Table 53 

Policy Outcomes: Teaching Varlf^bles, 1983-84 



Policies 


Sc 


100 1 


Syj 


3tem 


N 


Mean 


N 


Mean 


Management Is 


shared 


21 


A. 


lA 


10 


2.70 


Decisions are 


data-based 


70 


3. 


39 


10 


2.50 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent) 



Table 5A 

Procedural Outcomes: Teaching Variables, 1983-8A 



Procedures 



Inservlce modified to support SITIP 

Staff assignments and accountabili- 
ties modified 

Resources allocated annually 
Local funds used 



School 



N 



70 
70 

69 
18 



Mean 



3.41 
3.A7 

3.46 
3.33 



System 



_Sjrs 
N 



14 
10 

9 
8 



Mean 



2.50 
2.50 

3.78 
2.62 



Scale ranges from 1.00 (noi: at all) to 5.00 (to a very large extent) 
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Organizational outcomes were slight to moderate (Table 52), with the wStrongest 
(and most neutral) the establishment of status, and the weakest the sense of 
ownership. Policy outcomes were slight (Table 53), with particularly low 
ratings given to data^based decision-making in three LEAs where project 
leadership was school-based, coimnunlcatlon between school and central office 
Was infrequent » and (in two cases) harmony was poor,* Procedural outcomes 
(Table 54) were also low, with only respurce allocation above the moderate 
level. 

Low system level ratings put a greater burden on schools. Ratings given 
to the *'district-wide" LEA, for all factors except "ownership" and use of 
local funds, indicated that TV was built into operating procedures although 
still somewhat shaky. 

Relationships among outcomes . Data were grouped into the following 
indices: instructional gain, system institutionalization, school institution- 
alization, central office support, and school administrator siapport. 
Correlational analyses were conducted to determine the relationships among 
these Indices. 

Correlations among the five indices showed strong direct relationsaips 
(r - .50) between system institutionalization and central office support and 
between school Institutionalization and two other indices — school adminis- 
trator support and instructional gain. Instructional gain was positively 
related to school administrator support. (See Table 55.) 



It should be noted that after survey data were collected one LEA (with 
changing purposes, staff reassignments, and various other environmental 
turbulences) did make a major data-based decision — to terminate the^ 
project. In the circumstances this was appropriate. 



Table 55 

Intercorrelatlons Among Five Indices: Teaching Variables 





1 


2 


3 


4 


5 


1. 


Instructional Gain 




N.S. 


.60 


N.;s. 


.53 


2. 


System Institutionalization 






N'.S. 


.91 


N.S. 


3. 


School Institutionalization 








.71 


4. 


Central Office Support 








N.S. 


3. 


School Administrator Support 











N.S. ^ Not significant at the .05 level. 

Note: The number of cases upon which the correlations were 
calculated varied. 
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Influences and Plans 

Influences on past implementation and future action Included environ- 
mental turbulence such as: (1) staff changes resulting in shifts In SKIP 
leadership; (2) program changes reviHing priorities or restructuring tasks; 
and/or (3) ci»aul=ational changes. They also included the naturt, and «xtc".t 
of outside (MSDE) asdistanci-. , and Local perceptions of program success. 

Almost all jLEAs experienced Ihitiding cuts. However, in only one LKA did 
those cuts negat-Lvely influence SITIP In that central office staff raassign- 
ments led to reduced Involvemeat. Three other kinds of changes occurre': (1) 
to staff, resulting in shifts in leadership and decreases in program expertise 
(in schools of four LEAs, and also at the district office level in two of 
f those cases);* (2) to programs in four LEAs, in three caues slightly 

restructuring the use of TV and in the fourth combining TV with AT; and (3) to 
organizational arrangements in five LEAs that resulted in staff changes. 

Three LEAs experienced all three kinds of changes. In one, with a 
district-wide ^proach, no major negative impact was experienced, partly 
beqause only a few sites were affected. In the second, school-basejt leader- 
ship remained constant and was strong enough to overcome the potencial loi^s of 
energy indicated by the turbulence. In the third, the changes combined with 
previous setbacks to contribute to project decline. 

Turbulence created less of a problem when local perceptions of succes: 
were strpng. Such a perception was usually a combination of the opinions oi 
teachers (enjoying colleagial stratpgizing to improve instruction) and princi- 
pals (finding in TV a useful observation process or vehicle to bring staff 
together around instructional issues). Problems were sometimes alleviated 
somewhat by outside assistance, but this factor was much less significant. 



* In a third LEA staff changes at the district level increased expertise. 

17. 1 ISl 



When turbulence occurred and perceptions of success were slight, decIJnti was 
probable. 

By the end of Year 3, the LKA using a dlst rlct-wlde approach had used TV 
with seven of Its eight schools, and planned to continue incorporating "time" 
observation in supervision of reading, and add that component for mathematics. 
The five LEAs using other Impleraentat ion strategies had Involved 4,5% of their 
229 schools, and all but one planned to continue use of TV in Year A. Survey 
respondents indicated specific activities and influential factors, and lat^r 
(without reference to survey data) plans were submitted to MSDE by project 
coordinators. For most LEAs, individual responses and coordinators' plans 
agreed with each other. However, In one LEA survey respondents assumed 
continuation, but the coordinator decided to terminate the project after 
reviewing activities and outcomes. Survey responses are summarized in Table 
56. In general, most participants (51.4%) planned to maintain their current 
level of implementation, or allow voluntary use by teachers (35.1%), primarily 
because they belt /ed that teachers* instructional skills improved. Planned 
txpansion was within schools or to other classes or subjects, rather than to 
additional schools or models. Support will be somewhat reduced. Plans for 
reduction were influenced by funding cut*^ and relatively lew staff ^nipport. 
Also, in one LEA, termination was influenced by the beliei that almost no 
change had been made in teachers' behavior (although a great deal of efiort 
had been Invested ^n observations and record-keeping). 

Summary and Conclusions 

Over the three years, local involvement in TV Increased from six to 17 
schools, from 51 to 123 teachers, with six LEAs p.-jrt Icipat Ing In June 1984. 
In six elementary schools, teachers used TV for reading and/or mathematics; in 
11 secondary schools, TV was used for a variety of subjects. The "time" 
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Table 56 
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Activities Planned and Influences Perceived 
by Teaching Variables Implementers (N-74) 



Type of Activities 


% Respondents** 








X Respondents 


Per Factor* 












I 


2 


3 


4 


5 


6 


7 


8 


9 


10 


Maintenance 

Maintain current level 


51.4 


12.6 


6.1 


2.8 


15.4 


6.1 








i .4 




Allow voluntary use 


35.1 


1.1 


4.2 


4.2 


12.6 








4.2 




4.2 


Expansion 

Expand classes 


24.3 




2.8 


1.4 


15.4 


1.4 












Expand schools 


16.2 


1.4 




4.2 
















Add another component 
of model 


1.4 X 






















Support 

Conduct inservice 


12.2 








5.6 


1.4 






1.4 






Provide resources 


18.9 


2.8 


1.4 


1.4 


9.6 


1.4 


1.4 










Rpduc t inn 

Discontinue SHIP 


1.4 












6.9 


1.4 


6.9 






Reduce activity by 
25% or more 


1.4 












1.4 










Totals 




18.2 


14.5 


14.0 


58.6 


14.5 


9.7 


1.4 


12.5 


1.4 


4.2 



* Ten factors were listed on the survey, of which eight were indicated as influential in 
planning by some respondents, 

1. High student achievement data indicate SITIP value. 

2. SITIP helps achieve a local priority. 

3. Students and staff like SITIP. 

4. Teachers* inatruction.il skills improve with SITIP. 

5. Senior administrators advocate SITIP. 

6. Funding cuts: other things take priority over SITIP. 

7. SITIP Is not cost-effective 

8. There is little staff support for SITIP. 

9. Senior administrators have little Interest in SITIP. 
10. SITIP has very little to do with local priorities. 
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Many respondents checked more than one type of plan. 



variable was used more than "content/' and while fidelity of data collection 
was high, application of other components of the model varied among schools, 
and understanding of the concepts and strategies for improvement was low in 
some cases. 

In two LEAs, and one of the schools in a third district, teachers ran the 
project and conducted "time" observations. Administrative and supervisory 
staff conducted observations and provided leadership in other sites. In each 
site, a particular role group was primarily responsible for project 
management, with others providing support and apparently not challenging what 
was done; this may have contributed to Implementation that was incomplete or 
shallow in those sites where there was misunderstanding. 

Impact on teachers was perceived to be greater than impact on students. 
Overall instructional gain was low with no significant differences among LEAs 
or types of schools. Organizational^ policy, and procedural outcomes were 
moderate and plans for the 1984-85 school year Indicated maintenance rather 
than expansion, with the project being terminated In one LEA. School institu- 
tionalization was correlated with principal support and instructional gain. 
District institutionalization was correlated with central office support. The 
mean rating across LEAs for the latter was relatively low. 

Over the three years, ^ each district experienced some form of environmental 
turbulence, and was Influenced by the nature and extent of support (including 
training) provided by MSDE and TV developers. Negative influences 
(experienced by at least two LEAs) included: (1) initial developer training 
that was intense and particular, focusing on data collection rather than on 
Improvement strategies; (2) assistance from MSDE that attended more to process 
than to the model and the concepts of effective instruction; (3) asslntance 
from the developer that repeated the "time" training, advocated an application 
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to administrators and supervisors with one-on-one feed'back of data (a switch 
from the initial group strategizing approach), and provided minimal help to 
secondary schools; (4) changes in project purpose or organization that 
contributed to complexity, resistance, and confusion; (5) staff reassignments 
resulting in loss of leadership expertise; and (6) poor understanding or 
little request for information by local educatdrs resulting in incomplete or 
superficial use of the model. Positive influences (experienced by at least 
two LEAs) included: (1) local use of the model (with fidelity) to accomplish 
an existing school priority; (2) acquisition, by LEAs, of information and 
training to clarify or expand the model knowledge base (from the dev^oper and 
MSDE) ; (3) stability and expertise of local leadership, with press for 
fidelity and attention to application of improvement strategies; and (4) clear 
stable purpose, acceptable to teachers. 

In one district, combined forces contributed to project termination.* 
Decline possible in two other LEAs where administratlve^support Is low. It 
is probable that the level of implementation apparent in June 1984 in the 
seven schools where TV is used district-wide, and in five other schools, will 
become routinely integrated into school activities. However, in two schools 
with teacher observers, reduction in resource allocations would jeopardise 
implementation. 

While in Year 3, MSDE advocated local ownership, it may be advisable for 
TV project teams to review progress and request additional assistance for 



Local purpose did not facilitate TV fidelity^ staff reassignments (five) 
resulted in .oss of administrative expertise, and project modification was 
too late and with insufficient press for fidelity and completeness to be of 
value. Educators wanted success, provided resources, and worked hard and 
sincerely, but achieved virtually nothing. The decision to terminate ir> in 
keeping with MSDE*s recommendation to make data-based decisions, not 
allowing continuation of poorly Implemented projects. 
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Year A. Such assistance might give strong attention to classroom Improvement 
~ real changes made by teachers — and the "fit'' of local purpose with the 
model, how it is used, and resulta to date. While some schools (in Calvert 
and Montgomery) experienced some success, many did ngt, and if greater 
instructional gain is not apparent, school institutionalization is unlikely. 

Review, Across Models 
This section summarizes and compares findings across the four instruc- 
tional models. Discussion focuses on planning; the scope, intensity, and 
fidelity of use; the roles and responsibilities of implementers; the outcomes 
in terms of impact on stuaents, teachers, schools, and school systems; and 
influences and plans for Year 4. Some conclusions are drawn and recommenda- 
tions made. 

Planning 

Table 57 presents the status of objectives in June 1984 across all SITIP 
projects. The number of projects for each model addressing a given objective 
is stated. The percent of projects (across models) and the degree of achieve- 
ment is also stated for each objective. Improvement in student achievement 
was addressed by most projects, with 45% claiming accomplishment, and 48% 
considering the objective partly achieved. Partial or full achievement of 
objectives relating to teacher competence and classroom organization (//5 and 
#7) was claimed by 96% and 100%, respectively. Results were influenced not 
only by successful implementation, but also by the extent of expansion, since 
the greater the expansion the less total achievement of project objectives 
could be claimed. In general, progress was made over the three years, and 
plans developed by most LEAs were modified annually reflecting the relative 
success of previous years. 



Table 57 . ' 



Status of Local C^Jectlves: All Models, June 198A 







Number of Projects 
Addressing Objectives 


Percent of Projects 






Total 


AT 


ML 


STL 


TV 


Achie^ 


ing Objectives* 






N-30 


N-9 


N-8 


N-6 


N»7 


1 


2 


3 


. 1. 


Improve student achievement 
(basic skills) . 


•27 


8 


7 
« 


6 


6 


7 


A8 


A5 


2. 


Improve student achievement 

fnt'hpT' fliih'ippf'fli- 


22 


6 


5 


5 


6 


23 


A5 


32 


3. 


Inform local educators kbout 

ffn ri /I a 1 




7 


7 


o 


A 


1 1 


31 


58 


A 


TrA4n pHiiPAfnrfl Abouti models 


26 


8 


6 


5 


7 




50 


35 


5. . 


Improve teachers* classroom 
competence • 


Ik 


7 


6 


5 


6 


A 


63 


33 


6. 


Ensure match of instruction, 
curriculum, and testsfs). 


20 


A 


6 


5 


5 


0 


45 


55 


7. 


Help teachers become better 
organized • 


26 


8 


6 


5 


7 


0 


65 


35 




Improve time-on- task. 


Ik 


8 


5 


A 


7 


A 


58 


38 


9. 


Improve students' involvement 
in learning (motivation). 


25 


7 


6 


6 


6 


8 


52 


AC 



* l-Hoped for 193 
2=*Partly achieved 
3'Achieved 
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Scope, Intensity, and Fidelity of Use 

Influenced by the stAtegy of Implementation used and by administrators' 
Investment of time and Interest, the dimensions of scope, intensity, and 
f4dellty indicate the nature and extent of use. 

Scope . The scope of implementation by LEAs in June 1984 is presented in 
Table 58 and is summarized in Table 59. The 23 of 24 LEAs in the state 
receiving SITIP funds are listed. Since several LEAs implemented more than 
one model, there were more than 23 projects. Since each LEA determined its 
own allocation cf SITIP funds, multiple projects within a district were not 
necessarily equally funded, nor given equal attention. The strategies 
presented relate to those employed in Year 3. The strategies used were 

* 

sometimes different from those originally planned, expanding In seven cases 
from a lighthouse school approach to a pilot/district approach, and in two 
cases reducing to a lighthouse approach. Expansion was Influenced by local 
success (usually as perceived by administrators). Reduction (or termination) 
occurred due to minimal Impaot of SITIP (usually Influenced by the processes 
the LEA elected to use and environmental turbulence) . All types of schools 
were involved, including two vocational-technical centers, ranging from a 
single school in one LEA to 33 schools in another. As few as four teachers 
were involved in a project to as many as 700. The number of students ranged 
from 113 to 22,594.* 

Overall, more than 74,000 students were Involved. The 182 schools 
monitored by the study represented about 16% of Maryland's schools. More than 
51% were elementary, usually involving students in grades 3 through 5. Both 



* In Charles and Calvert counties, only the classes at the pilot schools were 
monitored, so the number of students reported is probably much lower than 
occurred in practice. 
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Table 58 

Scope of Implementation by LEA: All Models, June 1984 





it 












LfiA 


Model 


Strategy 


#of Schools 


Type 


//of Teachers 


i» //of Students 


Allegany 


ML 


LS 


2 


0 


18 


■ 

350 


Anne Arundel 


ML 


LS 


1 


H 


5 


300 


Baltimore City 


ML 


PD 


28 


J/M,H 


606 


22,594' 


Baltimore County 


ML 


PD 


6 


E 


32 


1,094 


Calvert 


STL 


PD 


3 


E,J/M 


13 


375 c 




TV 


PD 


3 


J/M 


18 


468+* 


Caroline 


AT 


PD 


7 


E,J/M 


85 


2,695 


Carroll 


ML 


PD 


5 


J/M 


7 


700 


Cecil 


AT 


PD 


25 


E,J/M,H 


700 


13,000 


Charles 


STL 


CB 


IS 


E.J/M.H 


116 


650+* 


Dorchester 


. STL 


PD 


4 


■ E 


16 


425 


Frederitk 


TV 


LS 


2 


J/M 


U 


' 350 


Garrett 


AT 


LS 


3 


J/M,H 


20 


1,000 


Harford 


AT 


DW 


33 


E,J/M 


671 


18,650 


Howard 


ML 


PD 


■6 


E,J/M 


35 


- 1,500 


Krat 


TV 


DW 


7 


E,J/M 


52 


1,561 


Montgomery 


AT 


LS 


1 


E 


8 


250 




STL 


LS 


1 


J/M 


7 


350 




TV 


LS 


2 


E,J/M 


14 


400 . 


Queen Anne's 


STL 


CB 


2 [ 


J/M,H 


23 


800 


St* Mary's 


AT 


CB 


7 


E,J/M,H 


62 


1,500** 


Somerset 


AT 


LS 


1 


E 


10 


300 




TV 


LS 


2 


E,H 


12 


,420 


Talbot 


TV 


LS 


1 


0 


13 


250 


Washington 


AT 


LS 




No Data 








ML 


LS 




No Data 








STL 


C3 


No Data 






Wicomico 


AT 


DW > 


16 




154 


3,850 


Worcester 


ML 


LS 


1 


E 




240 




STL 


LS 


1 


E 


4 


113 



* At pilot middle school. 
** Includes some duplicates. 

Model ; AT«Actlve Teaching 
ML=^a3tery Learning 
STL=Student Team Learning 
TV=Teachlng Variables 

Type : E«Eleraentary school 

J/M« Junior high/middle school 
) H«Hlgh school 
0»Other 



Strategy ; LS«Llghthouse school 
PD«Pllot district 
DW=Dlstrict wide 
CB=Capaclty building 
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•Table 59 

Suflunary of Scope of Implementation: All Models, June 198A 



Model 



Active Teaching 



Projects 



31 



Schools 



N-* 



E'65 
S 28 
93 



51 



Teachers 



1710 



62 



Mastery Learning 



27 



E 10 
S 37 
0 2 
49 



27 



711 



26 



Student Team Learning 



20 



E 15 
S 11 
26 



14 



200 



Teaching .Variables 



20 



E 6 
S 10 
0 1 
17 



123 



Total 



29 



100 



E 96 
S 86 
0 3 
185 



100 



2744 



100 



* Three schools Ctwo elementary and one secondary) are implementing two 
models. 

** Eighteen teachers are implementing two models. 

School ; E « Elementary 
S * Secondary 
0 » Other 

Note ; No data available for Washington County (Active Teaching and 
Mastery Learning). 
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Junior/middle and senior high schools were'*^included in the 85 secondary 
schools. About 2,7AA teacher^p^sed one or more models: additional teachers 
were trained within LRAs and used SITIP ideas at their own discretion. A 
comparison across models Indicates that Active' Teaching and Master Learning 
were the most widaly used (impacting about 56% and 36%, respectively, of SITIP 
students), and Student Team Learning and Teaching Variables were the least 
widely used (impacting abQut A% art#5%, respectively, of SITIP students) • 
While the relative simplicity ^of AT facilitated expansion, complexity was not 
a deterrent for a successful model :^ Mastery Learning was used by 27% of the 
SITIP schools. 

Intensity . The average numbe^ of years that teachers were involved in 
SITIP was 1.6 for AT, 1.8 for ML, and 1.9 for STL and TV. (Mean times were 
depressed by the number of teachers in expansion sites.) During che 1983-8A 
school year, teachers used STL for an average of five months, ML and TV for 
seven months, and AT for close to nine months. AT and ML teachers used the 
models for a larger percentage of the^ in-class time (a^i average of 51% and 
A3%, respectively) than. teachers using (36%) and STL (19%). Consistent use 
facilitated instructional gain. 

¥i : ility . Each model required the implementation of certain components. 
(See Figure 2 for components and extent of fidelity.) More teachers (91%) 
implemented all critical components of the AT model than did th6 implementers 
of the other three models (ML — 62%, STL — 59%, TV — A6% 18% "con- 

tent"). With the exception of ML, which was second only to TV in complexity, 
the more complex the model was eo implement, the less the degree of fidelity. 
The degree of fidelity was also related to the extent of administrator "press" 
for fidelity of impleihentktlon. In those LEA^ where administrators encouraged 
and expected to see fidelity of implementation, more teachers implemented all 
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Figure : 

SITIP Fidelity of loplcttentatlon: All Models 1983-^4 



Models and Coaponencs of Student Ta«a Learning* 



STAD ^8 Uipleaentsd. 
'^CT Is lApleasnted. 
Jlgsj^rls lapt^aented. t 
TAI lIl^flipleMntsd. 

Each teas lnclu(^es a mix of kinds of students 

ion given criteria), 
atarlala are available for peer tutoring, 
tcaa practice, and Individual and toumalMnt 
qulxsea. 

Qulz/tournaaent acoraa relate to Individual 

and tana achleveaent. 

Peer tutoring takes plsce s grsst dssl. 

Successes are publlclxad. 
All five coaponants. 



/ 



Coapouents of Hastery Learning 



1. j Objectives are specified. 

2. Objectives are broken dovn In^o coaponent 
akllls. ' 

3. Curricula (texts, aaterlsls) sre aatched to 
objectives. 

4. IrtPtructlon natches curricula and objectives, 
5* Tests aatch objaatives. 

6. Tests Include itefts froa both lover, and . 
higher order thinking skills. 

7. A "no fsult*' formative test Is given for e^ch 
unit. 

8. "Corrective" end "enrlchnent" activities are 
given after foraative tests. 

9. Suanativs teat a are given at the end of each 
unit. 

lOf Records on lev#l of aastery are kept per 
class per «Kud€»nt per thjectlve. 
All' ten coyponents. 



Teachers lapleaentlng 



26 
22 
11 
0 

35 

31 

31 
33 
32 
23 



68 
58 
29 
0 

92 



82 

82 
87 
84 
59 



Teachers^ lapleaentlng 



68 


ICO 


65 


96 


64 


94 


63 


96 


67 


98 


56 


82 


61 


.90 




98 


66- 


97 


59 


87 


47 


62 



Variables snd Coaponents of T^hlng Vsrlablea* 



The **contant" variable la lapleaentad. 
The **tlBe" variable Is laplcaented. 

1. Tlae on taak data have been collected on ay 
claaa 

- by ay principal 

~*by a central office supsrvlsor 

-* by a taachsir. 

2. Obtarvatlon raaulta are dlacuaaad Id ataff 
aeetlnga and ve help each other find strstsglss 
to laprovs tiM-on*tssk. 

3. Observstlon rssults sre discussed by as snd t^e 
obssrver snd vs agrse on iaproveaent atrstsgles 
(If ttssdcd). 

4. Dsts collscted on ay clsss Indlcsted thst tlae- 
on-*tssk vss such that I did not nssd to aake 
chsages. 

5. t bavs Bade chsogss In ay clsss to laprove 
tiae-on-tssk. 

6. Dsts collsctsd on ay clsss Indlcstsd that tlae- 
on-taak lap roved. 

7. Va uaad the "content" variable, aatchlog 
currlculua, Inatnictloo, and the CAT. 

8. Por the ''content" variable, I kaav what ay 
atud«nta had bean tansht and vhat their teat 
acoraa vara for laat year. 

9. Por the "content" variable. I aodlfla4 
curriculua and/or inatnictlon. 

10. Por the "content" variable, I kept recorda of 
content covered by objective for each report 
period. 

All three "tiat" coaponanta (1 , 2 or 3 . 4 or 5) 
All four "content" coaponanta (7-*10). 



Coi|>onants of Active Teschlng 



HosMsrork review snd orsl questioning. • 

Skill review snd presentstlon of nsw aatarlal. 

Guided practlcs. 

Indspsndent prsctlce. 

HoMHurk ssslgnaents. 

Weekly, aonthly, end of unit reviews. 

All six coaponents. 



Teschers ti 



N-46 



24 
36 



26 
16 
27 

16 

31 

26 
24 
20 
21 

15 
16 



12 
21 
9 



leaantlna 



52 
78 



b1 
35 
59 



67 

57 
52 
43 
46 

33 
35 



20 
46 
18 



Teachera laplaaanting 



ii-90 



88 
90 
89 
87 
88 

82 



99 
97 
98 

92 
91 



For Student Team !.#'arnlng and Tearhlng Varlrfbles, unnumbered items represent specific irodeli or varlflble«5, Impleoenters cou'd use anv one or all aodali 



or variables. 
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components of the model* Such "press »" plus the support provided In Years 2 
and 3 9 probably facilitated the fidelity and Intensity of ML (overcoming the 
uncertain Implementation experienced In some LEAs In Year 1). 

Administrative Investment * The average amount of time Invested by a 
SITIP administrator during Year 3 was 21*05 daysy with means ranging from 
16*45 days for AT to 26.65 days for ML* While overall means differed little 
between the two administrative role groups » there were differences among 
models^ with central office staff Investing more time than school-based 
administrators only for AT. For three models » one administrative role group 
Invested^ on average^ about twice the time Invested by the other role group* 
(TV was the exception where Investments were similar). Activity areas » In 
order of priority allocation of time. Included: (I) Inservlce, (2) general 
support, (3) administration/communication, (4) monitoring/evaluation, and (5) 
dl^semlnatlon/expanslon. While Inservlce and support were top priorities for 
three models, top priorities for TV were administration/communication, and 
monitoring/evaluation. Time investments and priorities were influenced by the 
nature of the model and the scope of iuplementation* Results suggest that 
success (in terms of instructional gain or institutionalization) is 
facilitated by administrative Involvement in inservlce and general support* * 

In general, the quality of Implementation varied. While there were some 
exemplary sites for all models, there were other sites where fidelity was low, 
or application was sporadic* Poor implementation was characterized as 
infrequent use of a model, pro forma application of parts of a model, lack of 
actual change in classroom behavior, or isolated teachers carrying out an 
adaptation as best they could* Excellent implementation was characterized by 
definable changes in classroom behavior. Increased student tlme-on-task 
directly linked with aligned curriculum and quality Instruction, use of the 
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model regularly and/or for a complete unit or course, and data-based 
decision-making. Administrators were well informed, supportive, and expressed 
clear expectations of fidelity and intensity in the sites where better Imple- 
mentation occurred. 

Roles and Responsibilities 

The SITIP design encouraged participatory decisi on-mak.".ng and involvement 
of all three instructional role groups In an LEA. By the enc of Year 2, it 
was apparent that: (1) teachers involved in MSDE training activities some- 
times became instructional leaders, and all teachers involved in SITI^ needed 
time to develop materials to be used in support and assistance of implementa- 
tion; (2) school-based administrators involved in MSDE training activities 
were more committed than those trained by LEAs, and all needed to support 
teachers' efforts for success; and (3) central office staff, after MSDE 
training, determined their roles by the extent to which a model met local 
priorities, contributing most effort through Inservice or general support, but 
contributing relatively little (e.g., only administration) when a lighthouse 
strategy was used or a model was perceived as more teacher-centered. 

In Year 3, participants, particularly the LEA teams that initiated local 
projects, were aware of each other's relative success and the processes and 
factors that irhibited or facilitated that success. They were advised by MSDE 
to consolidate successes, make appropriate- revisions, and make data-based 
decisions to terminate or institutionalize as state funds were withdrawn.- 
Particular attention was to be paid to interactive support and leadership. 

Interactive support . Support among LEA participants included exchanging 
information and materials; providing training, coaching, and trouble-shooting; 
managing logistics; and recognizing successes. Support from MSDE and 
developers consisted prima vir/ of training, technical assistance, networking, 
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and trouble-shooting. Survey respondents' ratings of support are presented In 
Table 60. As in previous years, the effects of visibility (frequ'iucy and 
accessibility of interactions) were apparent, with higher ratings awarded to 
role groups more visible to teachers. When several role groups were fairly 
equjllly visible, expertise and affective and logistical support influenced 
ratings. In general, developers, who interacted very little with local 
educators, received the lowest ratings, although STL developers, who were the 
most visible, were rated slightly lower than ML developers. Overall, MSDE 
staff received the next lowest ratings (all above average) with the expertise 
of ML TAs dhd the locally-responsive networking style of the STL TAs being 
well-perceived. While school-based administrators were rated somewhat higher 
than central office staff overall, the range for the latter group across 
models was much wider than for the former. This indicated that gchool-based 
administrators played similar roles, regardless of the model, but roles played 
by central office staff differed by model In visibility and deiao"<itratlon of 
expertise and In affective and logistical support. Support by teachers was 
rated most highly overall » with lower ratings for AT (which was the least 
complex model) » and TV (which made the least demands on npn-observlng 
teachers) , and higher ratings for STL (which was teacher-led In many 
districts) , and ML (which was the most complex model and made considerable 
demands on teachers)* 

In comparison to Year 2» overall ratings for each role group were slightly 
lower, suggesting the diminution of energy which might be expected as Institu- 
tionalization occurs. Slight increases were awarded to teachers for STL and 
TV, and to central office staff fur KL» which related to extra investoients of 
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Table" 60 

Perceptions of Support Received: All Models, 1983-84 





Support Groups 




- 


Models 


N 


Teachers 


School 
Administrators 


Central Office 
Staff 


MSDE 


Developers 


Active Teaching 


112 


3.66 


3.76 


3.11 


3.05 


2.49 


Mastery Learning 


89 


4.07 


3.74 


3.93 


3.52 


3.15 


Student Team Learning 


48 


4.18 


3.73 


3.67 


3.33 


3.13 


Teaching Variables 


70 


3.78 


3.75 


2.92 


3.20 


3.26 


Totals 


319 


3.88 


3.75 


3.39 


3.25 


2.72 



Mean ratings rang, from a low of 1.00 (poor) to a high of 5.«0 (excellenC). 
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effort which they made. Below average ratings awarded to TV central office 
staff were related to the fact that was that that role group was Involved In 
the projects In only two of the TV LEAs. 

Administrative leadership « Affective an4-^ logistical leadership behaviors 
are presented In Table 61 » together with ratings assigned to central office 
staff and school^based administrators for each of the models. Overall ratings 
for central office staff ranged from 2.49 (press for fidelity) to 3.99 (demon- 
strate commitment). Overall ratings for school-based administrators ranged 
from 2.63 (press for fidelity) to 4.23 (demonstrate commitment). For both 
role groups t affeetlve behaviors were more evident than logistical behaviors » 
and organizational process behaviors. were more evident than. those related to 
"press." With tile exceptions of central office staffs* data-baaed decision- 
making, and school administrators' press for fidelity and Intensity, all 
ratings for leadership behaviors were lower for TV than for other models. For 
all models except ML, ratings were higher fpr school administrators than for 
central office staff. 

In all cases, affective leadership behaviors were above average. 

Logistical leadership behaviors relating to the organizational processes 
of provision of assistance, coordination of communication, and Implementation 
of data-based decision-making, were above average with exceptions for the last 
three behaviors for central office staff In AT, and for communication 
behaviors for TV. Press for fidelity and Intensity were above average only 
for AT by school administrators. 

An analysis of variance showed significant differences between the four 
models on central office support (see Table 62). TV had the lowest, and ML 
had the highest mean on this Index. 
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Table 61 

Administrative Leadership Behaviors: All Models, 1983-84 





Mean Ratings Assigned 


Behaviors 


To 


Central Office Staff 


To School Administrators 




AT 

N-9A 


ML 
N-81 


STL 
N-A3 


TV 
N-53 


all 
N-271 


AT 
N-lOO 


ML 
N-83 


STL 
N-A8 


TV 

N-60 


all 
N-291 


Affective 






















Demonstrate connltment 




A. 35 


3.93 


3.35 


3.99 


A. 55 


A. 17 


A. 00 


3.95 


A. 23 


Provide support 


3.45 


A. 06 


3.7A 


3.11 


3.61 


A. 12 


3.98 


3.82 


3.80 


3.97 


Logistical 






















Press for fidelity 


0 77 


2.6A 


2.16 


2. OA 


2.A9 


3.02 


2.72 


2.10 


2.23 


2.63 


Press for Intensity 


2.95 


2.69 


2.33 


2. OA 


2.59 


3.28 


2.80 


2.30 


2.32 


2.79 


Provide assistance 


3.39 


A. 16 


3.86 


3.00 


3.61 


3.80 


A. 09 


3.83 


3.62 


3.85 


Coordinate LEA 
communication 


2.97 


3.76 


3.59 


2.69 


3.25 


3.19 


3. AO 


3. AO 


3.00 


3.25 


Coordinate school 
communication 


2.81 


3.61 


3.37 


2.68 


3.11 


3.57 


3.55 


3. AO 


3.30 


3.A8 


Implement data-based 
declslon^-maklng 


2.76 


A. OA 


3.30 


3.88 


3.A7 


3.09 


3.71 


3.2A 


3.17 


3.31 


Scale ranges from I ,00 (not at all) 


to 5. 


00 (to 


a very large extent) 


• 









AT-actlve teaching; m.-Mastery Learning; STL-Student Team Learning; TV-Teathlng Variables. 
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ANOVA Rtflults for Central Office Support 



Factor 


N 




P 


. df 


p 


1. Model 


228 


3.30 


9.19 


3/224 


.001 


AT 


77 


3.12 








ML 


65 


3.77 


STL 


37 


3.46 


TV 


49 


2.84 


'2. Strategy 


228 


3.30 


10.14 


3/224 


.001 


Lighthouse school 


87 


2.86 








Capacity bulldlns 


2i 


3.19 


Pilot district 


91 


3.64 


Dlatrlct-wlde 


29 


3.64 



AT - Active Teaching; ML Mastery Learning; 

STL - Student Team Learning; TV - Teaching Variables 
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There also were significant differences between the four implementation 
strategies on this index (see Table 62). The lighthouse school strategy had 
the lowest mean, and the pilot/district and district-wide strategies had the 
highest means on central office support. 

There were no significant differences between models or strategies on 
school administrator support. Given the relative success of models and 
projects (in terms of instructional gain and organizational outcomes), it was 
clearly desirable for both role groups to put into practice to a large extent 
affective behaviors and logistical assistance, for school administrators to 
put into practice all other behaviors to some extent (at least average), and 
for central office staff to implement those behaviors to some extent if there 
were to be more than lighthouse school projects. 

"Press" indicated administrative expectations of fidelity and intensity, 
without which teachers could assume that it was acceptable for them to make 
little or no change. Low administrative press was related to low success and 
potential project decline. 

In several LEAs, leadert;hip was undertaken by teachers (with administra- 
tive support). In some cases, teams of key teachers conducted training and 
coachings and, for TV, conducted classroom observations. In other cases, 
individual teachers ran the project, usually in a single school, but in one 
case across the LEA (with release time to do so). The strongest leadership 
behaviors of teachers/leaders were provision of assistance (when they had been 
trained at MSDE events), and support (when they believed in the model and had 
release time to help their colleagues) . The weakest behaviors were coordina- 
tion among schools (when they 1^d Insufficient release time and little 
Influence on other schools), ^*nd press for fidelity (when they had low 
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expertise in the model or In influencing others) In order to be effectlve> 
teacher leaders had to have real expertise In the model and strong administra- 
tive support* 

Overall interactive support, and leadership were goodt and for most 
projects improvements were made over the three years • However > there were 
problems if reassignments refiulted in leaders who lacked expertise or commit- 
ment if central office st^ff functioned only as administrators, if 
principals had priorities addressed by activities very different from SITIP, 
or if teachers were expected to do most of the work with little support* In 
contrast, where project teams remained stable and project management tasks 
were shared, where leadership behaviors were above average, and whetie 
expertise in the model helped achieve an existing priority, implementation was 
smoother, impact was more evident, and institutionalization more probable* 

Outcomes 

Institutionalization of successful projects was the desirable outcome for 
Year 3, and indicators were identified to determine the extent to which that 
was occurring* In addition, impact on students and teachers was assessed to 
determine Instructional gain* 

Students * As indicated in Table 63, students enjoyed SITIP classes (with 
STL being most popular), increased their achievement (most obviously In MI.)f 
retained more of what was taught (most obviously for AT) > took somewhat more 
responsibility for their own learning (more so for AT), andt in general, 
behaved a little better (more so for AT)* Empirical data — summaries of 



* Multiple reassignments — several key staff changed in one year, or project 
leadership changed each year — resulted in loss of expertise and momentum, 
contributing to project decline. 
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Table 63 

Instructional Impact as Perceived by Survey Respondents: All Modeie, 1983-84 





Models 


Impact on Instruction 


AT 
N-124 


ML 
N-97 


STL 
N-54 


TV 
N-72 


TOTAL 
N-347 


Instructional Value 












Works In the classroom. 

Is worth the work It takes. 


4.50 
4.29 


4.35 
4.05 


4.33 
3.96 


4.14 
3.82 


4.36 
4.14 


Impact on Teachers 












.Teachers enjoy it. 
Teachers have Increased knowledge. 
Teachers have Increased skills. 


4.06 
4 01 

4.05 


4.02 
4.19 


4.02 
3.77 


3.60 

'X OA 

3.92 


3.94 
4.02 


Impact on Students 












s 

Students enjoy It. 

Students' achievement has Increased. 
Students are leamlng/retalnlpg more. 
Students' general behavior Is better. 
Students are taking more responsi- 
bility for their own learning. 


3.99 
3.87 
3.09 
3.70 

3.i)6 


4.13 
4.04 
3.79 
3.55 

3.73 


4.47 
3.69 
3.63 
3.60 

3.77 


3.46 
3.47 
3.38 
3.58 

3.31 


4.00 
3.81 
3.72 
3.62 

3.39 



Mean ratings range from 1.00 (strongly disagree) to 5.00 (strongly agree). 



AT - Active Teaching; ML - Mastery Learning; STL - Student Team Learning; 
TV - Teaching Variables. 
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results of standardized tests and analyses of student progress comparing SITIP 
classes and non-SITIP classes — supported educators* perceptions that student 
achievement was significantly higher when AT or ML was Implemented i 
particularly in mathematics. No standardized test data were provided for STL 
or TV. 

Teachers. Teachers* knowledge of effective teaching and skill In 
Instruction Improved. 

Impact on teachers and students combined was defined as Instructional 
gain. A one-way analysis of variance showed significant differences between 
the four models on Instructional gain (see Table 64). TV differed from the 
other models on this index. Results showed that the mean for TV: (1) had the 
largest deviation xrom the total group mean, and (2) was the only model mean 
lower than the group mean. 



Table 64 

ANOVA Results for Instructional Gain: All Models 



Factor 


N 


X 


F 


df 


P 


1 . Model 


335 


3.93 


4.65 


3/331 


.003 


AT 


124 


4.00 








ML 


93 


4.01 








STL 


51 


3.96 








TV 


67 


3.67 








2. School Type 


320 


3.92 


8.85 


1/318 


.003 


Elementary 


160 


4.03 








Secondary 


160 


3.81 









AT - Active Teaching; ML « Mastery Learning; 

STL - Student Team Learning; TV « Teaching Variables 
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There were also significant differences between elementary and secondary 
schools on Instructional gain. The mean on this Index was significantly lower 
for secondary schools than for elementary schools* (This may have been 
Influenced by the fact that SITIP models were more often used In the latter 
for basic skills.) 

Schools . As Indicated In Table^^BS, educators agreed that the SITIP 
models worked In the classroom (with AT most strongly affirmed), and that they 
were worth the work they took (with strongest agreement apparent for AT). 

School organizational outcomes were fairly good, although local 
"ownership'* was only moderate (see Table 65). Policy outcomes (Table 66) 
Indicated that shared management and data-based decision-making were more 
apparent for SITIF than for other programs. Procedural outcomes (Table 67) 
Indicated that modification of Inservlce and staff assignments were occurring 
to a graater extent than allocation of resources and use of local funds. 
However, other data sources Indicated that most LEAs made significant In-klnd 
contributions. 

The three sets of outcomes — organizational, policy, and procedural — 
made up the Indicators for school Irybtltutlonallzatlon. An analysis of 
variance showed that; there were no Significant differences between models or 
strategies for school institutionalization. 

A multiple regression analysis was conducted to determine which indices 
(instructional gain, central office support, school administrator support, or 
fidelity) were the best predictors of school institutionalization.* Together, 
the four Indices explained approximately 48% of the variance in school 
institutionalization, which was significant at the .05 level. The strongest 

* Seventy-three teachers responded to all five Indices and were Included in 
the calculation. 
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Table 65 

A 

Organizational Outcomes: All Models, 1983-84 



4 

Oil t pomAs 


School 
N-310 


y o V. *i III 

N-55 


Cognitive 






Status of SITIP established 


3.76 


3.83 


Close to 100% of teachers asked 
CO parLXCipaLe ao so reguiany 


3.61 


3.35 


Affective 






Local educators feel "ownership" 
of SITIP 


3.27 


3.24 


There Is harmony between teachers 
and school^based administrators 
" about SITIP 


3.87 


3.78 


There Is harmony between school- 
based and central office staff 
about SITIP 


3.63 


3.95 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent) 
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Table 66 

Policy Outcomes: All Models, 1983-84 







School 


System 


Policies 




N 


Mean 


N 


Mean 


Management Is 


shared 


76 


3.96 


58 


3.50 


Decisions are 


data based 


313 


3.49 


57 


3.51 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent). 



Table 67 

Procedural Outcomes: All Models, 1983-84 



Procedures 


S< 


:hool 


85 


^stem ' 


N 


Mean 




Mean 


Inservlce modified to support SITIP 

Staff asslgments and accountabili- 
ties modified 

Resources allocated annually 
Local funds used 


309 

. 317 

315 
70 


3.75 
3.73 

w.3.50 
,3.07 


64 

' 57 

57 ' 
51 


3.61 
3.35 

3.68 
3.15 



Scale ranges from 1.00 (not at all) to 5.00 (to a very large extent). 
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predictor of schoQl Institutionalization was school administrator support, 
followed by instructional gain (see Table 68). School institutionalization 
was also strongly correlated with central office support (see Table 69). 



Table 68 

Multiple Regressibn Results for School Institutionalization 



Index 


B 


F 


School administrative support 


.4928 


27.320* 


Instructional gain 


.2365 


5.614* 


Central office support 


.1183 


1.36s 


Fidelity 


.0732 


.563 



- .47884 
Overall F - 17.538* 
N - 73 
*p ■ less, than .05 



School system . Institutionalization ^.ndlcators of outcomes relating to 
organization > policy, and procedures at the system level are presented In 
Tables 65, 66, and 67, with ratings given only by administrative and super- 
visory staff. In comparison to the school level, ratings assigned for the 
system level for the status of SITIP and for school and system harmony were 
somewhat higher. Also slightly higher at the system level were data-baeed 
decision-making, allocation of resources, and use of local fimds. Of some 
. concern were the extent of teacher participation, local ownership, modifica- 
tion of staff assignments, and use of local funds (although the ratii gs on the 
latter were somewlat misleading given the considerable investments of in-kind 
contributions). It should be noted that system-level outcomes were less 



important In LEAs focusing on a lighthouse school approach. However, from an 
overall cost-effective perspective, higher ratings were desirable since they 
indicated greater likelihood of district-wide institutionalization. 

Corr tlation among the five indices showed strong direct relationships 
(r - .50) between system institutionalization and two indices — central 
office support and school institutionalization (see Table 69). 



Table 69 

ANOVA Results for System Institutionalization 



Factor 


N 


7 


F 


df 


P 


' • Strategy 


A8 


3.60 


10.76 


3/AA 


.001 


Lighthouse school 


12 


2.51 








Capacity building 


4 


3.73 








Pilot/district 


2A 


4.01 








District-wide 


8 


3.95 




L_ . 





An analysis of variance showed significant differences between the four 
implementation strategies on system institutionalization (see Table 70). 
Results showed that the mean for the lighthouse school strategy: (1) had the 
largest deviation from the total group mean, and (2) was tlie only strategy 
mean lower than the group mean. 

There were no significant differences between the models on system 
institutionalization. 
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Table 70 

Intercorrelatlon Among the Five Indices: All Models 



Index 


1 


2 


3 


4 


5 


1. 


Instructional Gain 




.44 


.34 


.23 


.26 


2. 


System Institutionalization 






.67 


.80 


.42 


3. 


School Institutionalization 








.51 


.68 


4. 


Central Office Support 










-.45 


5. 


School Administrator Support 













Note ! The number of cases upon which the correlations were calculated varied. 
Influences and Plans 

Table 71 presents activities planned and Influences perceived across all 
SITIP models. Maintenance and minimal expansion (to other classes or subjects) 
were the most prevalent activities, with 57.35Z respondents planning the 
former, and 36.93Z planning the latter. Expansion by adding components (e.g., 
for STL or TV) or models was not likely (4.67% and 2.60%, respectively), and 
neither was reduction (2.33% suggesting termination and 2.65% suggesting a 25% 
project reduction). 

The single strongest Influence on plans was student achievement results 
(62.91%), followed by Improvement In teachers' Instructional skills (49.65%). 
Achievement of a local priority and liking for SITIP were about equally 
Influential. The strongest negative Influences were funding cuts, with other 
things taking priority over SITIP (Indicated by 3.85% of the respondents), and 
lack of staff support (2.97%). Less than 10% of the local respondents identi- 
fied negative influences on any planned activities. 

In addition to the influences discussed above, three kinds of environ- 
mental turbulence led to changes during implementation which influenced 
relative success. In some cases, funding cuts stimulated the changes. 
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Table 71 

Activities Planned and Influences Perceived by Implementers: All Models (N-344) 



Type of Activity 


X Respondents 
Planning* 


X Respondents Indicating Influential Factors** 






1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


Maintenance 

Maintain current level 
Allow voluntary use 


57.35 
21.80 


26.81 
14.39 


7.04 
2.05 


10.35 
6.26 


13.73 
5.06 


5.76 






.90 


.30 


.90 


Expansion 
Expand - classes » 
Expand - schools 
Add component 
Add A new model 


36.93 
12.80 
4.67 
2.60 


10.31 
2 .04 
1.75 
1.17 


3.57 
2 . 36 
11.69 
.27 


7.85 

1 / /. 
1 .44 

.60 


11.29 
2 .02 
1.16 


.87 
1 .78 
.60 
.27 




.28 








Support 

Conduct Inservlce 
Provide resources 


17.45 
22.07 


2.03 
4.41 


2.61 
2.33 


1.17 
2.35 


7.91 
8.48 


1.14 
2.03 


.30 
.60 


.28 


.30 






Reduction 
Discontinue SITIP 
Reducte activity by 25X 
or more 


2.33 
2.65 












1.78 


.60 


1.77 






Totals 




62.91 


31.98 


30.02 


49.65 


12.45 


3.85 


1.16 


2.97 


.30 


.90 



to 
o 



* Many respondents checked more than one type of plan. 



** Influential Factors: 



1. High student achievement data Indicate SITIP value. 

2. SITIP helps achieve a local priority. 

3. Students and staff like SITIP. 

4. Teachers' Instructional skills Improve with SITIP. 

5. Senior administrators advocate SITIP. 

6. Funding cuts: other things take priority over SITIP. 

7. SITIP Is not cost-effective. 

8. There Is little staff support for SITIP. 

9. Senior administrators have little Interest In SITIP. 
10. SITIP has very little to do with local priorities. 
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However, even when that was so, the negative impact was reduced when staff 
commitment and instructional gain were high enough to stimulate the extra 
effort needed to overcome cutbacks and setbacks. The three kinds of changes 
were: (1) staff reassignments resulting in shifts in leadership (which were 
negative when expertise was lost or communication lags occurred); (2) program 
changes revising priorities (often positive when SITIP was expanded, sometimes 
negative if administrators involved teachers in multiple projects); and .(3) 
organizational changes (negative when SITIP leaders received additional 
assignments or participants had to learn new systems). When two or more kinds 
of changes occurred and perceptions of SITIP value were low, the project was 
likely to decline. However, even when high environmental turbulence occurred, 
projects survived and made progress when scores were high on the five key 
Indices of successful implementation: system institutionalization, central 
office support, school administrator support, school institutionalization, and 

instructional gain. 

In Year 4, the indicators of administrative support and of institution- 
alization may be used by local educators in self-assessment, and efforts 
Invested as needed to make improvements if projects are to be effectively 
integrated into existing systems. If there is little or no Instructional 
gain, it might well be appropriate for projects to be terminated. 

In Year 4, 21 LEAs will receive state funds to support maintenance and 
expansion of SITIP projects. In most cases (exceptions include Baltimore City 
and Baltimore Count^) , new schools will not be added, but additional teachers 
may become involved and current participants will receive additional training 
and support. To date, it appears that local educators have given SITIP a fair 
trial, with some LEAs contributing a great deal more than they felt they 
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received frotn MSDE. Where their experience Indicated that the SHIP models 
did not meet their needs (or their Implementation strategy and processes did 
not facilitate success), LEAs did not expand, and three terminated. Where 
instructional gain was apparent and administrative leadership and support 
helped accomplish outcomes relating to organizational, policy, and procedural 
outcomes, institutionalization was more likely. Overall, all LEAs benefited 
in some way from their involvement. 
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SUMMARY AND CONCLUSIONS 



Each of the two preceding chapturs has Included a summary of findings, 
and various sections of the report have provided background Information. 
Rather than repeat those discussions, this chapter attempts to answer 
questions most often posed by researchers, policy makers, and practitioners 
who are Interested In large scale Instructional Improvement. Most such 
questions are contained In the overall question: 

• If the "bottom line" Is Instructional gain, and If that Is 
accomplished by bringing about long-lasting, worthwhile changes In 
teachers' behavior, what are the processes and content to be applied 
by large systems such as state departments or large school districts? 

Since SITIP was Informtd by the research on classroom and school effectiveness 

and planned change* the authors of this report believe that the processes » 

findings* and conclusions ate generalizable* and may well prove to be useful 

to those Involved in similar projects elsewhere. 

If an Instructional Improvement program Is Initiated, what Indices should be 
monitored to gauge the "health" of the program and the probability of eventual 
Institutionalization ? 

• Indicators of program fidelity and Intensity Include: 

1. The extent to which participating educators carry out critical 
components of the program regularly and/or continuously. 

• Indicators of Instructional gain Include: 

1. Impact on teachers: Increase In knowledge and skills, positive 
attitude to the program. 

2. Impact on students: Increase In achievement, learning, and 
retention; Improvement In general behavior and the extent to which 
they take responsibility for their own learning; a positive 
attitude toward the program. 
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• Indicators of administrative support Include:* 

1. Affective behaviors that... 

(a) demonstrate commitment and belief In the program's value 

(b) provide support by demonstrating Interest and recognizing 
teacher success. 

2. Such logistical behaviors as... 

(a) a "press'' for fidelity^ monitoring Implementation, and 
expecting a given level of use of the program 

(b) a "press" for Intensity, monitoring Implementation, and 
helping to ensure that at least three teachers In each 
participating school use the program regularly 

(c) providing assistance by coordinating, training, responding to 
requests, and providing resources 

(d) coordinating communication across hierarchical levels for 
program review and Improvement 

(e) Implementing data-based declslon*maklng. 



• indicators of Institutionalization Include: 

1. Organizational outcomes 

t 

(a) cognitive: the status of the program Is commonly understood, 
clearly stated, and close to lOOZ of teachers asked to 
participate do so regularly 

(b) affective: local educators feel "ownership" of the program; 
there la harmony between teachers and school-based adminis- 
trators about the program; and there Is harmony between 
school«based staff and central office staff about the program. 

2. Policy outcom<is 

(a) management (leadership, advocacy, decision-making) is shared, 
not reliant on a single administrator 

(b) effectiveness is assessed and data are used in decision- 
making. 



* Tasks that are specifically administrative (e.g., budget) take minimal time, 
and are subsumed under 2c of administrative support. 
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3. Procedural outcomes « 

(a) Inservlce Is modified to support the program 

(b) staff are assigned and accountabilities are modified 

(c) resources (timet materials) are allocated annually 

(d) local funds are used. 

• The strongest predictors of school Institutionalization are support 
from school-based administrators and Instructional gain. This 
Indicates that the program selected has to be one that really makes a 
difference In the classroomt and Is sufficiently linked to the 
principalis priorities to Influence administrative Investment In 
affective and logistical leadership behaviors. 

e Dlstrlct-vlde Institutionalization Is strongly correlated with central 
office support and schooWevel Institutionalization. The lighthouse 
school Implementation strategy does not facilitate dlstrlct-^lde 
Institutionalization. H6vever» central office support Is more evident 
when pllot/dlstrlct or dlstrlct-vlde approaches are usedt and In 
programs with high probability of Instructional gain. 

e If the program selected has proven Its value elsewhere » but results la 
little or no Instructional gain at a new site, the fidelity and 
Intensity of use should be assessed. If both are high but apparent 
for only a few Isolated teachers » administrative support needs to be 
improved and organizational » policy » and procedural outcomes assessed 
and modified if institutionalization is to occur. 

How can a state education agency (or large school system) use a relatively 

small amount of money to facilitate instructional Rain ? 

In all LEA-^EA interactions* the SEA should be a supporter or 
facilitator of instructional Improvement » acting on the assumptions 
that the SEA may Influence but cannot control the LEAt and the 
immediate responsibility for ijistructional change rests with the LEA. 

Within the SEA> particularly in the early jphases of project design^ 
interactive strategic planning should be conducted to establish a 
clear common knowledge base of local Interests » state expectations t 
and relevant research-abased alternatives. 

The SEA can offer grants to participating LEAs» requiring matching 
funds after the first year^ continuing funds after the third year only 
for expansion of local efforts » and expecting LEAs to attend to 
program weaknesses identified in annual evaluation reports. 

The SEA should sponsor a series of related training activities » 
carefully designed to link relevant research and exemplary practice, 
pre«*contraptlng with LEAs so that expectations and responsibilities 
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are clear, coaching researcher-presenters to meet participants* needs, 
Involving LEAs teams as presenters to publicize successes, conducting 
''matching" training actlvltlea for state staff and faculty of colleges 
and universities Involved In pre-servlce, and putting Into practice 
concepts and components of the Joyce and Showers training model 



• The SEA should use existing organizational mechanisms and modify staff 
responsibilities for planning and delivery of services, rathef/than 
creating new structures or totally reassigning staff. 

• The SEA should provide technical assistance to LEAs by ,esti^blishlng a 
cadre of technical assistants (TAs) with a strong knowledge base, 
on-golng communication and learning opportunities, and time, adminis- 
trative support, 'and professional capability to help LEAs carry out 
the plans designed by local teams. 

• The SEA should encourage cost-effective Implementation designs that 
build state Initiatives Into existing local priorities. ^ 



If the Innovation Is directly related to a local priority, how can /an LEA 



• A cross-hierarchical team should be formed that takes responsibility 
for planning, making decisions, and modifying activities by using 
information «bout the relative effectiveness of the program. This 
team (plus other representatives of role groups) should/ have a 
thorough understanding of the innovation so that plans jare realiatrlc 
and policy and practice are interactive. j^' 



e A pilot/district or district-wide strategy can be used. In both 
cases, the goal Is for all program eligible teachers io be 
implementing the model regularly by the end of the tt>ird year, and a 
process of incremental involvement is used. The pil^t /district 
strategy begins by focusing on a very few schools, and expands by 
school, beginning in each school with the principal's support and an 
active team of volunteer teachers. The district-wide strategy begins 
by focusing on teachers (from all schools) with responsibility for a 
given subject area and grade level(s), and expands by grade level (and 
sometimes also by subject area) . In both cases, awareness training 
should be conducted for all administrative and supervisory staff 
before teachers are trained. "First wave" participants should be 
volunteers to the extent feasible. 



See Joyce, B.R., 4 Showers, B. Power in staff development through research 
on training . Alexandria, Va.: ASCD, 1984. Components of the model include: 
rationale and theory building (for awareness) , demonstration and modeling 
(for conceptualization), practice and feedback (for skill development), ori- 
site coaching (for application or horizontal transfer), and Integrated 
learning (for executive control or vertical transfer). 






• Attention should be paid to the Indicators of Institutionalization > 
particularly those -relating to organizational outcomes and administra- 
tive support. 

• Participating teachi^rs should be given release tlme> and school teams 
should have common planning time In their first year of Irapleraentatlon, 
with more time available if curriculum materials are to be developed. 

• Classroom Instruction should not begin until teachers are prepared to 
teach a complete unit or course. Stops and starts, sporadic Implemen- 
tation, and low fidelity should be discouraged by team leaders 
providing relevant coaching or support so that participating teachers 
can experience success. 

• Assessment of Implementation processes and Instructional gain should 
be on-going to Inform decisions — replicating successes, and dealing 
with problems as soon as they are Identified. 

Hov can an SEA maintain Inter-agency harmony and encourage^ high productivity ? 

• The SEA should employ cross-hierarchical and Incer-agency planning and 
decision-making. 

• The SEA should put into practice a philosophy of assistance rather 
than monitoring, and of developmental assessment to make Improvements 
rather than after-the-fact evaluation or pro forma review. 

t The SEA should establish clear expectations for local projects and 
provide relevant assistance, information, and training to help LEAs 
meet those expectations. The TA should "press'* for fidelity and 
intensity of implementation. 

■> 

• The SEA should acknowledge local efforts, publicize successes, and 
facilitate networking. 

• The SEA TAs should identify problems in their learly stages, and 
alleviate or tesolve them effectively, dealing with individual or 
organizational conflict and responding quickly to requests for help. 

• The SEA should maintain program integrity, fostering beliefs and 
behaviors that facilitate instructional gain, open communication, 
and constructive problem-solving. 

• The SEA should acknowledge the realities of failure by helping L^As to 
terminate unsuccessful projects gracefully, rather than letting them 
slowly fade away. 

• The SEA should acknowledge the interests of the various educational 
groups, such as colleges of higher education, and design activities to 
exchange information and/or explore opportunities for collaboration. 

9 
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• The SEA should encourage real involvement by all role groups, 
including LEA superintendents , central office staff, school-based 
administrators, and teachers in exchanging information about program 
challenges and successes. 

• The LEA leadership teams should develop plans through which 
involvement in the state initiative maintains program fidelity and 
also serves a local priority. The ^EA should suggest revision of 
plans (or non-involvement) if local objectives are inappropriate to 
the innovation* 



How can technical assistants be most useful in Improving classroom instruction? 

• If Staff are assigned as part-time technical assistants (TAs), they 
need to fonoT a group that has strong leadership from a senior adminis- 
trator, and real support from their regular supervisors. The TA group 
coordinates program planning, communication, and resource allocation, 
and designs major awareness-level training activities. The group 
invests energy in achieving program goals (avoiding tangential or pro 
forma activities) . 

• Staff selected as TAs shpuld value the program and have regular 
assignments that can readily be integrated with the improvement 
project. Their responsibiljtties should be adjusted so that each TA 
team (of two or three people) spends 40 to 60 days a year on the 
improvement project* 

• Effective TAs should have a sound knowledge of instruction, curriculum 
development, staff development, planned change and organizational 
analysis, and the models/innovations to be implemented. They*^ should 
be familiar with schools and school systems, particularly those to 
they which are assigned, and should establish positive productive 
relationships with those systems, maintaining program integrity in the 
context of local constraints. 

• In order of time invested overall, the following tasks are addressed 
by TAs: (1) visiting sites to assist and review local implementation; 

(2) conducting training; (3) developing program activities and 
planning, at state and local levels; (4) administration and budget; 

(3) maintaining communication and exchanging, ideas within the TA , 
system; (6) building knowledge; (7) providing generaj, support to local 
educators; (8) disseminating; (9) selecting and developing materials; 
and (10) evaluation. In a three year effort, most TA time is needed 
in the second year. 

• Three recurring problems should be addressed by the TA group: (1) 
conflicting demands of regular and project responsibilities, (2) 
Individual opportunity to learn and group attention to effective 
communication, and (3) differences between state and local expecta- 
tions of the responsibilities of each organization and the "ownership" 
of the progra'Ji. 
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c From a local perspective* TAs are^seful when they are program 
advocates; provide <]uallty Infonpltlon* training* and assistance 
relevant to local, needs to facilitate rich fidelity of Implementation; 
and engage In cross-hierarchical problem-solving that helps to clarHy 
program purpose* maintain harmony* and contribute to instructional 
Improvement. TAs should support local leadership teams* and 
^ acknowledge successes. ^ 

How can training activities be most useful In Improving classroom Ijr/st ruction ? . 

• The ov^r|rll design for training should Include various kinds of 
activities for viarlous audiences. In general* trainers should' 
recognize that the more Intense the Intended outcome* the greater the 
frequency of trainer/trainee Interaction* and the smaller the ratla_jQl_^ 
trainers to tralne^^s. Also* the less intense the outcome (e.g«* 
awareness)* the l^ss Investment la likely from par. dpanta. 

• While participants enjoy the less liitense kinds of training activities* 
they value and are more likely to be'^inf luenced by the more Intenee 

^ activities (especially when content Is highly relevant) . As the 

project progresses^ training activities should Increase In Intensity* 
and when participants have "executive control" of the model/Innova- 
tion* training should stop, x 

• Training should be directly related to participants' need to know* 
building on existing knowledge* and addressing current tasks and 

* Interests. Pre-contracting clarifies mutual expectations of 

participants* responsibilities for action following training. 

• Training should be designed to be transferred* using a trainer of 
trainers approach* or expecting district and school teams to follow 
through on the more Intense activities. 

• Team training* with common activities for everyone as well as activi- 
ties for specific roles and for groups with varying levels of 
expertise* should address both program content and implementation > 
processes. 

• ^ Tralnlnff should be conducted by outside "experts*" such as program 

developers* and by state and local instructional leaders, with each 
event including trainers from several role groups. ("Outsiders" 
should be carefully coached about trainees' interests.) 

• Teachers w^ conduct training usually are most successful when they 
rMUai t'i'Wftu' I teams of two or three people. Individual teacher- 
ttalners need strong support from administrative staff. . 

• The , final components of training (on-site coaching and Integrated 
learning) can be facilitated by three-person teams in each school 
which are supported by ^ district-wide network. 
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• While the integrity of the knowledge base (program fidelity and 

completeness) should be maintained, trainers should be flexible in the 
delivery of training, modifying methods to meet participants' needs 
(e*g*, using different approaches for different schools). 



If the innovation (s) match local priorities, and the research on planned 

change is applied to facilitate implementation, what is the likelihood of 

local institutionalization of an externally-initiated program ? 

• The likelihood of some projects being institutionalized i<^ increased 
by the use of the research on planned change, but schools are impacted 
by the constant changes of society* Schools and school systems cannot 
accurately predict or control the social pressures or environmental 
turbulence that can change priorities or undermine programs. However, 
they can select innovations most likely to addtess the basic business 
of schooling — effective instruction — in which they naturally 
invest thei.* own efforts and for which external support may be a 
welcome temporary addition. 

In lighthouse schools. Institutionalization is probable if the 
principal's priorities are addressed, there are no conflicting 
innovations, and at least three teachers 'advocate and implement the 
innovation regularly. 

Institutionalization is unlikely if funds are used primarily for 
training (a capacity-building approach) with voluntary application by 
trainees, unless school teams pre-contract and those teams receive 
follow-up assistance in the context of an administrative "press" for 
implementation. 

Institutionalization is likely where the indicators (described 
earlier) are attended to from the beginning of the program, where 
materials development is essentially completed by the middle of the 
second year, and where total local ownership is expected by the end of 
the third year. 



If instructional gain is the intended outcome, what kind of innovation is best ? 

Instructional gain is defined as (I) increase in teachers' knowledge 
and skill in effective instruction and a positive attitude toward the 
programi an4 (2) increased student athlevement and acceptance of 
responsibility for their own learning, and a positive attitude towards 
the program. 

• Since instructional gain is such a comprehensive construct, it is 

unlikely to be achieved without careful implementation and planning, a 
reasonable scope and intensity of use, and application of an innova- 
tion designed to achieve such an outcome. The four models used in 
SITIP were so designed, and careful attention was paid to implementa- 
tion processes. 
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t Assuming appropriate Implementation and good fidelity » greatest 
Instructional gain Is likely If Active Teaching or Mastery Learning 
are U8ed> with best results in elementary mathematics > or for 
structured academic subjects Ir) secondary schools. Student Team 
Learning results In somewhat less Instructional gain since educators 
tend to use It sporadically. Least gain Is likely for Teaching 
Variables since educators tend to use It for assessment rather than 
for Improvement* 



Was Maryland's School Improvement Through Instructional Process program a 



• Yes It was. Furthermore^ It continues to be a success since a large 
number of schools are Involved > many educators have Increased their 
knowledge and skill In planned change and Inst rue tlorial Improvement, 
and many students have Increased their achievement, participation, and 
enjoyment through SITIP-based lessons. 
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